Skip to content
Marketing Factory Digital GmbH
Contact
Logo Marketing Factory Digital GmbH
  • Agency
    • About us
    • History
  • Services
    • Consulting, Analysis and Strategy
    • Programming and Development
      • Interface Development
      • PIM/ERP Links
      • Custom Development
      • Seamless CMS Integration
    • Hosting and Support
      • Operation on our Colocation Hardware
      • Cloud Strategies
    • Services with Third Parties
  • Technology
    • TYPO3
      • Current TYPO3 Versions
    • Shopware
    • IT Security
      • DDoS Protection
      • Continuous Upgrading
      • Privacy First
    • Tech Stack
      • Commitment to Open Source
      • Technology Selection
      • PHP Ecosystem
      • Containerisation & Clustering
      • Content Delivery Networks
      • Search Technologies
  • References
    • Projects
    • Clients
      • Client List
    • Screenshot of the homepage of the new Maxion Wheels websiteNEW: Relaunch of the corporate website of Maxion Wheels
  • Community
    • Community Initiatives
  • Blog
  • Contact
  • Deutsch
  • English

You are here:

  1. Blog
  2. Help me, TYPO3 Upgrade Wizard! How to properly migrate Gridelements records.
  • Development
  • TYPO3
10.03.2022

Help me, TYPO3 Upgrade Wizard! How to properly migrate Gridelements records.


The fourth and final part of this blog series: our new containers and content elements are configured and ready to use. But the old records created with grid elements and their child elements still exist in the project.

Therefore, this time we will show you how to migrate existing data.

Possible paths of data migration

In our specific project, we migrated data in three ways:

1. Using a simple SQL script

When it was only a matter of minor adjustments in the database, we made them via SQL scripts. For example, we renamed backend layouts during the upgrade; their identifiers then had to be updated in the page properties.

UPDATE `pages` SET `backend_layout` = 'pagets__1_column' WHERE `backend_layout` = 'pagets__7';
UPDATE `pages` SET `backend_layout_next_level` = 'pagets__1_column' WHERE `backend_layout_next_level` = 'pagets__7';

Similarly, we also replaced some layouts and frames with new solutions.

2. Manually create the new records

Of course, automation is always preferable. But let's face it: for a small number of records, this is faster than writing an update script. However, this approach is also not possible in every project.

3. Writing custom TYPO3 Upgrade Wizards

You might know Upgrade Wizards from the TYPO3 Install Tool. For example, when upgrading from TYPO3 v9 to v10, you need to migrate the records from the outdated "pages_language_overlay" table to the "pages" table.

TYPO3 provides an interface that allows developers to create their own upgrade wizards.

An Upgrade Wizard is suitable for complex migrations:

  • Creating new records in any table
  • Changing the CType
  • Copying old data into new tables (fields)
  • Updating FAL relations
  • Adjustment of field values
  • Deletion of old records
  • Restricting migration to specific CTypes, parent grid elements, ...

Our complete example below will show this very clearly.

Structure of a TYPO3 Upgrade Wizard

A well understandable tutorial for creating your custom Upgrade Wizards can be found in the official TYPO3 documentation.

Therefore, we only list a few key data here:

  • Own Upgrade Wizards can be added to the sitepackage or other extensions.
  • You can check if a migration is necessary before executing the upgrade.
  • The order in which the upgrade wizards are to be executed can be determined if necessary.

It may also help to have a closer look at other upgrade wizards from the TYPO3 core or extensions.

Show larger version for: Tab element
A group of tabs between which the user can browse by clicking (screenshot from www.bootstrap-package.com)

Example: Migration of Gridelements to the Tab element of the Bootstrap Package

The following exemplary Upgrade Wizard was written by my colleague Mirena Peneva.

The initial situation in the project:

  • A single-column "tab container" Gridelement.
  • Each of these Gridelements can contain several content elements of type "Text & Images" (CType "textpic")
  • Some content elements contain images (FAL relations)
  • In the frontend, the content elements grouped in this way are displayed as tabs

Our goal:

  • Using the tab element (with inline elements) from the Bootstrap Package.

Tasks:

  • Select all affected elements and change their CType accordingly.
  • Create new inline elements for the previous grid elements child elements and migrate existing content to them.
  • If available, link FAL relations in the "sys_file_reference" table to the new inline element
  • Delete old records

ext_localconf.php:

$GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['ext/install']['update']['tabs']
    = \MyProject\Sitepackage\Updates\MigrateTabs::class;

Classes/Updates/MigrateTabs.php:

<?php

namespace MyProject\Sitepackage\Updates;

use Exception;
use InvalidArgumentException;
use RuntimeException;
use TYPO3\CMS\Core\Database\ConnectionPool;
use TYPO3\CMS\Core\Database\Query\QueryBuilder;
use TYPO3\CMS\Core\Database\Query\Restriction\DeletedRestriction;
use TYPO3\CMS\Core\Utility\GeneralUtility;
use TYPO3\CMS\Install\Updates\UpgradeWizardInterface;

/**
 * Migrates existing "Tab Container" content elements to CType "tab".
 * In TYPO3 v8, the website contains a gridelements container with layout "Tab Container".
 * The contents of this container have to be migrated to use the CType "tab" for bootstrap package tabs.
 * "Tab Container" content elements are restricted to colPos "0" in all backend layouts.
 *
 */

class MigrateTabs implements UpgradeWizardInterface
{
    /**
     * @var int
     */
    protected int $gridElementBELayout = 14;

    /**
     * @var string
     */
    protected string $tableTab = 'tt_content';

    /**
     * @var string
     */
    protected string $targetTableTabItem = 'tx_bootstrappackage_tab_item';

    /**
     * @var string
     */
    protected string $targetCTypeTab = 'tab';

    /**
     * @var int
     */
    protected int $colPos = 0;

    /**
     * @var string
     */
    protected string $sourceFieldNameImage = 'image';

    /**
     * @var string
     */
    protected string $targetFieldNameImage = 'media';


    /**
     * Return the identifier for this wizard
     * This should be the same string as used in the ext_localconf class registration
     *
     * @return string
     */
    public function getIdentifier(): string
    {
        return 'tabs';
    }

    /**
     * Return the speaking name of this wizard
     *
     * @return string
     */
    public function getTitle(): string
    {
        return 'Tabs: Migrate existing tabs with Grid Layout "Tab Container"';
    }

    /**
     * Return the description for this wizard
     *
     * @return string
     */
    public function getDescription(): string
    {
        return 'Migrate existing tabs with Grid Layout "Tab Container"';
    }

    /**
     * Execute the update
     *
     * Called when a wizard reports that an update is necessary
     *
     * @return bool Whether everything went smoothly or not
     */
    public function executeUpdate(): bool
    {
        $this->migrateTabs();
        return true;
    }

    /**
     * Migrate existing content elements to new CType "tab".
     * 1. Get all gridelements containers with Grid Layout "Tab Container" and change their CType to "tab".
     * 2. Add a new content element for each of the container's children.
     * 3. Migrate the fields "header", "bodytext" and "image" to the fields in the new elements:
     *    "header", "bodytext" and "media"
     */
    protected function migrateTabs()
    {
        $connection = GeneralUtility::makeInstance(ConnectionPool::class)->getConnectionForTable($this->tableTab);
        $connectionPool = GeneralUtility::makeInstance(ConnectionPool::class);
        /* @var QueryBuilder $queryBuilder */
        $queryBuilder = $connectionPool->getQueryBuilderForTable($this->tableTab);
        $queryBuilder->getRestrictions()->removeAll()->add(GeneralUtility::makeInstance(DeletedRestriction::class));
        try {
            // 1. Get all gridelements containers with Grid Layout "Tab Container"
            $gridElementsContainers = $queryBuilder
                ->select('*')
                ->from($this->tableTab)
                ->where(
                    $queryBuilder->expr()->eq('tx_gridelements_backend_layout', $this->gridElementBELayout)
                )
                ->orderBy('uid')
                ->execute()
                ->fetchAllAssociative();

            foreach ($gridElementsContainers as $container) {
                // and change their CType to "tab"
                $fields = [
                    'CType' => $this->targetCTypeTab,
                    'tx_gridelements_container' => 0,
                    'header' => 'Tab Container',
                    'header_layout' => 100,
                    'colPos' => $this->colPos
                ];

                $updatedRows = $connection->update(
                    $this->tableTab,
                    $fields,
                    ['uid' => $container['uid']]
                );

                if ($updatedRows > 0) {
                    // Get all children elements in the selected gridelements containers
                    $tabItems = $queryBuilder
                        ->select('*')
                        ->from($this->tableTab)
                        ->where(
                            $queryBuilder->expr()->eq('tx_gridelements_container', $container['uid'])
                        )
                        ->orderBy('uid')
                        ->execute();

                    foreach ($tabItems as $item) {
                        // 2. Add a new content element for each of the container's children
                        $queryBuilderTabItem = $connectionPool->getQueryBuilderForTable($this->targetTableTabItem);
                        $insertSuccessful = $queryBuilderTabItem
                            ->insert($this->targetTableTabItem)
                            ->values([
                                'pid' => $item['pid'],
                                'tt_content' => $item['tx_gridelements_container'],
                                'header' => $item['header'],
                                'bodytext' => $item['bodytext'],
                                'tstamp' => $GLOBALS['EXEC_TIME'],
                                'crdate' => $GLOBALS['EXEC_TIME'],
                                'mediaorient' => 'right',
                                $this->targetFieldNameImage => $item[$this->sourceFieldNameImage]

                            ])
                            ->execute();

                        if ($insertSuccessful) {
                            // Migrate existing images
                            $newItemUid = $queryBuilder->getConnection()->lastInsertId();
                            $fileReferenceQueryBuilder = $connectionPool->getQueryBuilderForTable('sys_file_reference');
                            $fileReferenceQueryBuilder
                                ->update('sys_file_reference')
                                ->where(
                                    $queryBuilderTabItem->expr()->andX(
                                        $queryBuilderTabItem->expr()->eq(
                                            'fieldname',
                                            $queryBuilderTabItem->quote($this->sourceFieldNameImage)
                                        ),
                                        $queryBuilderTabItem->expr()->eq('uid_foreign', $item['uid']),
                                        $queryBuilderTabItem->expr()->eq(
                                            'tablenames',
                                            $queryBuilder->quote($this->tableTab)
                                        )
                                    )
                                )
                                ->set('uid_foreign', $newItemUid)
                                ->set('tablenames', $this->targetTableTabItem)
                                ->set('fieldname', $this->targetFieldNameImage)
                                ->execute();
                        }
                    }

                    // Delete old tab items
                    $queryBuilderTabItemsOld = $connectionPool->getQueryBuilderForTable($this->tableTab);
                    $queryBuilderTabItemsOld
                        ->delete($this->tableTab)
                        ->where(
                            $queryBuilderTabItemsOld->expr()->eq('tx_gridelements_container', $container['uid'])
                        )
                        ->execute();
                }
            }
        } catch (Exception $e) {
            throw new RuntimeException(
                'Database query failed. Error was: ' . $e->getMessage(),
                1605857008
            );
        }
    }

    /**
     * Is an update necessary?
     *
     * Is used to determine whether a wizard needs to be run.
     * Check if data for migration exists.
     *
     * @return bool
     */
    public function updateNecessary(): bool
    {
        $updateNeeded = false;
        if ($this->checkIfWizardIsRequired()) {
            $updateNeeded = true;
        }

        return $updateNeeded;
    }

    /**
     * Returns an array of class names of prerequisite classes
     *
     * @return string[]
     */
    public function getPrerequisites(): array
    {
        return [];
    }

    /**
     * Check if there are gridelements containers with matching Grid Layout.
     *
     * @return bool
     * @throws InvalidArgumentException
     */
    protected function checkIfWizardIsRequired(): bool
    {
        $connectionPool = GeneralUtility::makeInstance(ConnectionPool::class);
        $queryBuilder = $connectionPool->getQueryBuilderForTable($this->tableTab);
        $queryBuilder->getRestrictions()->removeAll()->add(GeneralUtility::makeInstance(DeletedRestriction::class));

        $numberOfEntries = $queryBuilder
            ->count('uid')
            ->from($this->tableTab)
            ->where(
                $queryBuilder->expr()->eq('tx_gridelements_backend_layout', $this->gridElementBELayout)
            )
            ->execute()
            ->fetchOne();

        return $numberOfEntries > 0;
    }
}

This Upgrade Wizard is a helpful blueprint that you can adapt to your project. Some experience with the TYPO3 QueryBuilder is an advantage, but it is also a good opportunity to get familiar with it.

We wish you a lot of success trying it out! Do you have any questions?

Sebastian Klein

Standing somewhere between the front-end and back-end. With a soft spot for usability and documentation. Always on the lookout for good practices.

More posts by this author

Get blog posts as RSS feed

All parts of this blog series

  1. Mistakes of the past: how we systematically replace Gridelements
  2. Planning is half the migration: What to consider when replacing Gridelements
  3. Improving TYPO3 Container elements
  4. Help me, TYPO3 Upgrade Wizard! How to properly migrate Gridelements records.

Please feel free to share this article.


Comments

No comments yet.

Write a comment.

I have been informed that the processing of my data is on a voluntary basis and that I can refuse my consent without detrimental consequences for me or withdraw my consent at any time to Marketing Factory Digital GmbH by mail (Marienstraße 14, D-40212 Düsseldorf) or e-mail (info@marketing-factory.de).

I understand that the above data will be stored for as long as I wish to be contacted by Marketing Factory. After my revocation my data will be deleted. Further storage may take place in individual cases if this is required by law.

  • Data privacy policy
  • Legal notice

© Marketing Factory Digital GmbH

Picture Credits
  1. "Article image Gridelements migration": © Sebastian Klein / Marketing Factory Digital GmbH
  2. "Tab element": www.bootstrap-package.com