Translations
Info
All page names need to be in English.
en da  de  fr  it  ja  km  nl  ru  zh

CMS WV Code Sprint 2013 Frankfurt

From TYPO3Wiki
Jump to: navigation, search

About

The conceptional meeting and code sprint to revive workspaces & versioning will take place in Frankfurt/Main, Germany.

Hashtag

We did not decide on a hashtag for this event but #t3sprint seems appropriate here, too.

Date and Times

Tuesday, June 11th - Wednesday, June 12th 2013

We'll start the meeting around 10am on both days. We can use the meeting room (and other facilities) at dkd on both days as long as we need them.

Map and Location

dkd Internet Service GmbH will be host of the code sprint. The conference room is preserved for the code sprint on both days.

Address: Kaiserstraße 73, 60329 Frankfurt/Main, Germany

Map: http://goo.gl/maps/u4BZ5

Code Sprint Topics/Goals

  • Clear up conceptional and technical questions
  • Define ways for proper testing in this area and start a test base
  • Fix already recorded bugs
  • Record (and possibly fix) bugs not yet publicly known (i.e. in Forge)
  • Work for a vision of and a strategy for the further development of workspaces & versioning

Participants

(Please add your name and the days you plan to participate)

  • Thorsten Kahler (both days, accommodation needed)
  • Sascha Egerer (both days, accommodation needed)
  • Jo Hasenau (both days, no accomodation necessary since I will be in Darmstadt anyway)
  • Uwe Trotzek (both days, accommodation needed)
  • Lienhart Woitok (both days, accommodation needed)
  • Timo Webler (both days, no accomodation needed)
  • Chris Zabanski (both days, no accommodation needed)
  • Anja Leichsenring (only Tuesday, no accommodation needed)
  • Danijela Grgic (only online, no accommodation)
  • Tymek Motylewski (only online, no accommodation)
  • Oliver Hader (both days, accommodation needed, single-room)


dkd is taking care of travel and accommodation expenses for all participants up to 150 € per day and participant.

Accommodation

Recommendation:

TODO

Sponsoring

  • dkd is sponsoring unlimited coffee (ok, limited due to race conditions during office hours)
  • citeq is sponsoring food and drinks

Contact

If you have questions, or if you like to participate or offer some sponsoring, just get in contact with Thorsten Kahler (thorsten dot kahler at typo3 dot org).

Thanks

Thanks go to

  • dkd for usage of their conference room and sponsoring of travel expenses and accomodation
  • citeq for sponsoring of food and drinks


Proceedings

The first day was filled with tough but fruitful discussion about the basic principles of workspaces + versioning. Pinning down the underlying DB changes for different processes (think: CRUD) on the whiteboard was a big help to clear our minds and achieve a common understanding. This was the base for rethinking the currently known issues with relations of any kind: one-to-one, one-to-many, many-to-many; between records, as IRRE relations and even for translation handling.

There were two cases that required some deeper thoughts:

  1. Translations
    Versions of translation records are constructed by different kinds of overlays. Because the i18n mode for every DB field can be configured individually the translation overlay has to be merged in a sophisticated process. Overlaying the current (workspace) version has to be done before this merge.
  2. Many-to-many relations
    In theory mm relations can be seen as a table that mainly consists of two 1:m relations. From this perspective adding version fields to mm tables would be the natural approach.

Adding versioning to mm tables would either mean a major breaking change or having to parse all DB queries and adjust mm joins to select the correct version. Both solutions didn't seem promising and left us with a more or less sleepless night.

The second day was planned to get some hands dirty in code. So we split into two groups: one to elaborate on testing opportunities, the other to dig deeper into the mm stuff and advance the documentation. Not surprising both groups didn't finish their task - but both did a great leap forward.

Regarding the mm relations we found out that we had to leave off of a unified theory: if mm relations are handled differently than other relations (where the foreign key always points to the uid of the live record) there's no need to introduce a breaking change or "manually" rewrite all SQL joins. The trick is to duplicate all relations when versioning a record and let the duplicate point to the new version. The duplicate then belongs to a workspace through either the local or the foreign side. When using enableFields() and other API methods (be it Extbase or TYPO3\CMS\Core\Database\RelationHandler aka t3lib_loadDbGroup) properly the relations belonging to versioned records are filtered out in existing code. (At least they should but these would be changes on bugfix level if required.) Up to here that's the case already. There's one new rule that has to be ensured by \TYPO3\CMS\Core\DataHandling\DataHandler aka t3lib_TCEmain (or any 3rd party code that creates versioned records): when creating a new version in a workspaces it has to be checked whether a mm relation already exists in that workspace; if so, it must not be duplicated but the foreign key pointing to the live record has to be changed to point to the workspace version of the record. (TODO: add a link to the rules.txt)

Building a reliable test base was the aim of the second group. To not only get a common understanding but also reproducible tests they created a Vagrant box on Github and added a set of test cases in the accordant wiki. The Git repo also contains a file and DB dump (based on the introduction package) to kickstart the TYPO3 instance and a test extension embedded as Git submodule which gradually can be enriched with all kinds of data relations, translation modes etc. possible in TYPO3.

Showing the results of both groups to each other and discussing them made the second day perfect.

The yield of the two day meeting in Frankfurt is a solid base for further distributed work on the improvement of TYPO3s workspaces and versioning features. Next steps will be to shape the draft results to be included in the different documentations and an Extbase code sprint in July or August with a focus on workspaces and versioning.

--Thorsten (talk) 12:19, 18 June 2013 (CEST)

The new rules for workspaces in TYPO3 CMS

So actually the code sprint was not that much about coding but about discussing possible solutions and the impact they might have in different scenarios. Finally we managed to define 5 major rules that have to be respected when coding workspace aware methods for TYPO3 CMS extensions or the core.

Rule I

Any workspace related action, except those handling records of MM tables, is always based on the UID of the live record. This is true for the transOrigPointerField during translations as well.

This means:

As soon as an editor starts working on any kind of record within a workspace, there MUST be either an original default version or a placeholder record in the live workspace, so that the workspace record can point to this parent record.

Rule II

If the transOrigPointerField field of any record is not empty it can point to UIDs of default language records in the live workspace only. We have to make sure that only those records can be selected as parents of a certain translation, that haven't got a translation of that language already.

This means:

If the translation in the live workspace is not connnected to any parent record in the default language, this will be true for the workspace version as well. Setting a parent record within a workspace action will be possible, but then again the transOrigPointerField field can point to default language records of the live workspace only, and it can point only to those that haven't got a translation of that language within the same workspace already. This has to be checked to avoid double translations of the same record.

Rule III

Any MM related workspace action will trigger a copy of any affected record of the MM table into the workspace, with the uid_local or uid_foreign being replaced with the UID of the newly created workspace version. There is one exception for this rule: If there already IS a workspace version of that record in the MM table that points to the UID of the affected live record, this record of the MM table will not be copied anymore but updated with the UID of the newly created workspace version instead.

This means:

There are 4 different kinds of MM records possible in the same table:

  1. uid_local -> live, uid_foreign -> live => this is possible in a live workspace only
  2. uid_local -> WS, uid_foreign -> live => this is possible in a non live workspace only
  3. uid_local -> live, uid_foreign -> WS => this is possible in a non live workspace only
  4. uid_local -> WS, uid_foreign -> WS => this is possible in a non live workspace only

This way we can make sure to be able to recognize any workspace related MM record for any particular workspace easily while joining tables. And it does not matter from which side of the relation we are going to start, so bidirectional relations will work as well. We don't have to work with additional fields, since records will be unique within a workspace and just must be different from 1. And we can make sure as well to be able to add and/or remove any MM record within a workspace without ever changing the result of the live workspace. We can rewrite core methods under the hood so they will be able to recognize these changes, while the method calls won't change. So we can reduce the impact on extensions to those that are using their own database queries instead of core methods. But since workspaces and MM relations were buggy anyway, the impact should not be too high.

Rule IV

Translations make it necessary that we will ALWAYS make a workspace overlay of any translated record before.

This means:

There are different possible combinations of overlays giving you the result for a language workspace preview.

  1. Default -> WS, Translation -> live
    1. Get default record
    2. do the default WS overlay.
    3. Get translated record.
    4. Overlay 1.2 with 1.3
  2. Default -> live, Tranlation -> WS
    1. Get default record
    2. Get translated record.
    3. do the translated WS overlay.
    4. Overlay 2.2 with 2.3
  3. Default -> WS, Translation -> WS
    1. Get default record
    2. do the default WS overlay.
    3. Get translated record.
    4. do the translated WS overlay.
    5. Overlay 3.2 with 3.4
  4. Default -> N/A, Translation -> WS
    1. Get translated record.
    2. do the translated WS overlay.

Conclusions

Now it is recommended to check if your extensions are already compliant to these rules, and if not, to provide updates making them at least compatible to TYPO3 6.2 LTS. We are planning to provide backported versions of these changes for officially supported versions of the TYPO3 CMS core, but not as official bugfix versions. This is due to the fact that most of the TYPO3 instances out there are not affected by these problems anyway and we want to protect them from possible side effects. So if you want to provide working versions of your workspace aware extension and still be supporting versions before 6.2, you should wait until we were able to provide patched versions of the core. Of course there will be patches available for testing via the usual channels on forge.typo3.org and review.typo3.org

We hope this solution will give a maximum of benefit for those who are using workspaces while lowering the impact on the rest of our users.