Translations
Info
All page names need to be in English.
en da  de  fr  it  ja  km  nl  ru  zh

DocTeam/ReST migration

From TYPO3Wiki
Jump to: navigation, search
This page belongs to the Documentation Team (category DocTeam)


Goals

This project was born out of discussions about the need to make TYPO3 CMS and Flow/Neos documentation converge. TYPO3 CMS (= v4/v6) documentation is still based on OpenOffice, whereas Flow/Neos documentation is based on ReST. Furthermore the designs used are not the same, which is not good for users, nor for the TYPO3 brand.

Migrating the TYPO3 CMS documentation away from OpenOffice had been a long-standing idea. The fact that ReST is already used by the Flow/Neos documentation and the need to converge with the latter has set this project into motion again.

The goals of this project are thus:

  • to migrate all the official documentation of TYPO3 CMS to ReST
  • to define a common design for TYPO3 CMS and Flow/Neos manuals
  • to adapt the Flow/Neos manuals to the common format and design
  • to define a number of ReST transformations providing professional documentation for TYPO3 and related projects
  • tell the exact line number when a validation error of a manual occurs

Pros and cons

The decision to revive the migration to ReST for TYPO3 CMS documentation was preceded by a public debate in the mailing lists in September 2011.


Redefined milestones

Below are redefined milestones, which contain the stuff, which has not yet been done. These milestones can be worked on simultaneously. The remaining 15 man days of budget have been distributed to these milestones as noted below.

Milestone 1: TYPO3, TER & typo3.org integration

Budget: 4 days

Deadline: September 2012

Make the appropriate changes to TYPO3 (mostly the Extension Manager and if possible also the "tsconfig_help" module) to handle ReST format for the system extensions manuals, and also link to the documentation server to browse the online documentation or download PDF versions. For the TER, the links to the documentation must be updated to use the new documentation server. The same goes for the documentation pages on typo3.org, which link to various manuals. Additionally it is desirable to display documentation "transparently" on typo3.org, one way or another (current ideas are full integration or usage of a proxy).

Note: this work will be split with the budget for TER work, as proposed by Fabien Udriot.

Milestone 2: Workflow automation

Budget: 5 days

Deadline: August 2012

Develop a work queue with the appropriate triggers for rendering documentation when needed. This needs to work for TYPO3 extensions, Flow packages, TYPO3 Core releases (for system extensions) and official documentation. This would ideally use Flow's REST API (note, that's REST as in "Representational state transfer", not ResT as in "ReStructured Text"). Choose and implement the formats to render the documentation to for the first release.

Milestone 3: Content storage and landing pages

Budget: 3 days

Deadline: August 2012

Use Flow to drive the new documentation site. A domain model must be defined for storing information about each rendered manual. Landing pages must be designed to be able to list specific manuals.

Milestone 4: Output tuning and integration

Budget: 3 days

Deadline: October 2012

Finalize the design of the documentation as rendered in HTML format, also design templates for PDF output. More generally the documentation site must match the look and feel of typo3.org. HTML output must also be prepared for Solr indexation. Solr search must be included in the site.

Milestones

Below are a number of first milestones (as of October 2011) that represent a first organization of the project into several large steps. This should probably be improved. Some steps can be performed - or at least started - in parallel. (The information above is more current and contains only the things, which are still missing as of July 2012.)

Milestone 1: Create a sample manual

references: https://git.typo3.org/Documentation/TYPO3/Example/Manual.git?a=tree;f=OpenOffice;h=4e76dafdc03e732403c172ca23629b92ceb89719;hb=b916a9435b044cfe24e4b210a3616511ccc1cbdd

Martin's rendering of this manual in reST: http://mbless.de/4us/typo3-oo2rest/The-TYPO3-Sphinx-Theme/build/html/doc_official_documentation_template/official_documentation_template.html

This finishes step 1.

Milestone 2: Create the rendering scripts

Most things are easy to do.

Problem: Complex tables

Idea to use a directive "list-tables": Write content as unordered lists => transform them into tables.

Even better: "field-list-tables". Creates a better structure in the reST-document. Fabien Udriot is working in that direction, too (related to the BLE projects).

Intermediate result: Getting the field-list-table directives integrated into the sphynx core is more difficult than thought. To avoid having to patch the Sphynx Core for the TYPO3 Core Docs we proceed without these directives. For extension manuals, however, they still are a good possibility.

Milestone 3: Convert the main TYPO3 CMS (= v4/v6) manuals to reST

Several ways to convert OOffice documents.

1) use XSLT transformation for converting the underlying XML

2) open a OOffice file, convert to HTML and write reST with docutils by adding a reST writer to docutils

Martin B. favors method 2.

First results can be seen here: https://docs.typo3.org/typo3cms/. This has already been checked in into the official Git Repositories at git.typo3.org.

Milestone 4: Make the needed changes to TER, typo3.org and the EM so that the new manuals can be displayed and downloaded

Have HTML, maybe with comments

Maybe also for extensions (optionally)

Milestone 5: Provide an environment for producing reST

Do what's possible to improve the user experience of writing reST. Ideas include:

1) have a service where people can upload their reST file and test the rendering (to avoid having to install Sphinx on their computer) Current thoughts are in the mailing list especially here.

2) contribute improvements to http://rst.ninjs.org/ (open source, see: https://github.com/anru/rsted)

Budget

This project benefits from a budget allocation from the TYPO3 Association, thanks to a budget request made by François Suter.

The estimate for the budget were based on the above milestones:

  • Total: 31 man-days

This total is not expected to cover all the work to do (far from it), but to help compensate participants for their efforts and pay for some of the harder or more boring tasks ahead.

Tasks

Most of the actual tasks for the many steps to take have not yet been defined. There are however a couple of early tasks that are up for grabs:

  • Inquire into print-on-demand:
    • what providers are there out there? They should be global players to ensure that TYPO3 manuals can be bought world-wide. Note: one well-known such provider is lulu.com.
    • what are the (financial) conditions of these providers?
    • are there any particular technical issues to consider when preparing a file for print-on-demand?

Brainstorming and notes

Tools

See Editors_(reST).