Laravel 5.4升级,从utf8转换为utf4mb

时间:2017-03-09 20:16:53

标签: php mysql database laravel encoding

我正忙着将我的一个网站从5.3升级到Laravel 5.4。我注意到,在浏览当前的Github存储库时,默认字符集和排序规则已从utf8更改为utf8mb4,以便为表情符号提供支持。

我当前的数据库(MariaDB 10.0.29)目前设置为使用utf8,但我想升级它以使用utf8mb4。不幸的是,我还没有找到有关此过程的任何文档。

也许我过度思考它,但我认为改变数据库的字符集和整理需要一些工作,至少运行一些ALTER TABLE命令。

  • 是否有必要更改数据库和表格上的字符集和排序规则?或者仅仅更改我的config/database.php文件中的设置就足够了吗?
  • 如果是这样,任何人都可以提供一个示例迁移(或一些MySQL代码)来说明如何实现这一点,同时要记住保留现有数据至关重要。

由于

2 个答案:

答案 0 :(得分:6)

好吧,我已经编写了一个迁移来实现我自己的系统。

  • 它允许您选择指定连接名称以引用默认连接以外的连接。

  • 它使用SHOW TABLES查询从连接数据库中获取表格列表。

  • 然后循环遍历每个表并将所有字符串/字符类型列更新为新的字符集和整理。

  • 我已经做到这一点,因此必须提供回调以确定列是否应将其长度更改为提供的新长度。在我的实施中,长度大于191的VARCHARCHAR列会在上移期间更新为长度为191,并且更新长度为191的VARCHARCHAR列会更新反向/向下迁移时长度为255。

  • 一旦更新了所有字符串/字符列,将运行几个查询来更改表的字符集和排序规则,将任何剩余的排序规则转换为新排序规则,然后更改默认字符集和整理表格。

  • 最后,数据库的默认字符集和排序规则将被更改。

注释

  • 最初,我试图简单地将表转换为新编码,但遇到了列长度问题。在我的MySQL / MariaDB版本中使用InnoDB时,utf8mb4中的最大字符长度为191个字符,并且更改表格排序会导致错误。

  • 我起初只想将长度更新为新长度,但我也想提供回滚功能,所以这不是一个选项,因为在反向方法中我会设置长度utf8mb4到255的列,这些列太长了,所以我也选择更改排序规则。

  • 然后,我尝试更改varcharchar列的长度,字符集和整理时间太长,但在我的系统中,当我有多个时,这会导致错误包含此类列的列索引。显然,多列索引必须使用相同的排序规则。

  • 重要提示就是反向/向下迁移对每个人来说都不是100%完美。我不认为在迁移时不存储有关原始列的额外信息就可以这样做。因此,我当前的反向/向下迁移实现是假设长度为191的列最初为255.

  • 同样重要的注意事项就是这会盲目地将所有字符串/字符列的排序规则更改为新排序规则,而不管原始排序规则,因此如果有列不同的校对,它们都将被转换为新的校对,反之亦然,原件不会被保留。

<?php

use Illuminate\Database\Migrations\Migration;

class UpgradeDatabaseToUtf8mb4 extends Migration
{
    /**
     * Run the migrations.
     *
     * @return void
     */
    public function up()
    {
        $this->changeDatabaseCharacterSetAndCollation('utf8mb4', 'utf8mb4_unicode_ci', 191, function ($column) {
            return $this->isStringTypeWithLength($column) && $column['type_brackets'] > 191;
        });
    }

    /**
     * Reverse the migrations.
     *
     * @return void
     */
    public function down()
    {
        $this->changeDatabaseCharacterSetAndCollation('utf8', 'utf8_unicode_ci', 255, function ($column) {
            return $this->isStringTypeWithLength($column) && $column['type_brackets'] == 191;
        });
    }

    /**
     * Change the database referred to by the connection (null is the default connection) to the provided character set
     * (e.g. utf8mb4) and collation (e.g. utf8mb4_unicode_ci). It may be necessary to change the length of some fixed
     * length columns such as char and varchar to work with the new encoding. In which case the new length of such
     * columns and a callback to determine whether or not that particular column should be altered may be provided. If a
     * connection other than the default connection is to be changed, the string referring to the connection may be
     * provided as the last parameter (This string will be passed to DB::connection(...) to retrieve an instance of that
     * connection).
     *
     * @param string       $charset
     * @param string       $collation
     * @param null|int     $newColumnLength
     * @param Closure|null $columnLengthCallback
     * @param string|null  $connection
     */
    protected function changeDatabaseCharacterSetAndCollation($charset, $collation, $newColumnLength = null, $columnLengthCallback = null, $connection = null)
    {
        $tables = $this->getTables($connection);

        foreach ($tables as $table) {
            $this->updateColumnsInTable($table, $charset, $collation, $newColumnLength, $columnLengthCallback, $connection);
            $this->convertTableCharacterSetAndCollation($table, $charset, $collation, $connection);
        }

        $this->alterDatabaseCharacterSetAndCollation($charset, $collation, $connection);
    }

    /**
     * Get an instance of the database connection provided with an optional string referring to the connection. This
     * should be null if referring to the default connection.
     *
     * @param string|null $connection
     *
     * @return \Illuminate\Database\Connection
     */
    protected function getDatabaseConnection($connection = null)
    {
        return DB::connection($connection);
    }

    /**
     * Get a list of tables on the provided connection.
     *
     * @param null $connection
     *
     * @return array
     */
    protected function getTables($connection = null)
    {
        $tables = [];

        $results = $this->getDatabaseConnection($connection)->select('SHOW TABLES');
        foreach ($results as $result) {
            foreach ($result as $key => $value) {
                $tables[] = $value;
                break;
            }
        }

        return $tables;
    }

    /**
     * Given a stdClass representing the column, extract the required information in a more accessible format. The array
     * returned will contain the field name, the type of field (Without the length), the length where applicable (or
     * null), true/false indicating the column allowing null values and the default value.
     *
     * @param stdClass $column
     *
     * @return array
     */
    protected function extractInformationFromColumn($column)
    {
        $type = $column->Type;
        $typeBrackets = null;
        $typeEnd = null;

        if (preg_match('/^([a-z]+)(?:\\(([^\\)]+?)\\))?(.*)/i', $type, $matches)) {
            $type = strtolower(trim($matches[1]));

            if (isset($matches[2])) {
                $typeBrackets = trim($matches[2]);
            }

            if (isset($matches[3])) {
                $typeEnd = trim($matches[3]);
            }
        }

        return [
            'field' => $column->Field,
            'type' => $type,
            'type_brackets' => $typeBrackets,
            'type_end' => $typeEnd,
            'null' => strtolower($column->Null) == 'yes',
            'default' => $column->Default,
            'charset' => is_string($column->Collation) && ($pos = strpos($column->Collation, '_')) !== false ? substr($column->Collation, 0, $pos) : null,
            'collation' => $column->Collation
        ];
    }

    /**
     * Tell if the provided column is a string/character type and needs to have it's charset/collation changed.
     *
     * @param string $column
     *
     * @return bool
     */
    protected function isStringType($column)
    {
        return in_array(strtolower($column['type']), ['char', 'varchar', 'tinytext', 'text', 'mediumtext', 'longtext', 'enum', 'set']);
    }

    /**
     * Tell if the provided column is a string/character type with a length.
     *
     * @param string $column
     *
     * @return bool
     */
    protected function isStringTypeWithLength($column)
    {
        return in_array(strtolower($column['type']), ['char', 'varchar']);
    }

    /**
     * Update all of the string/character columns in the database to be the new collation. Additionally, modify the
     * lengths of those columns that have them to be the newLength provided, when the shouldUpdateLength callback passed
     * returns true.
     *
     * @param string        $table
     * @param string        $charset
     * @param string        $collation
     * @param int|null      $newLength
     * @param Closure|null  $shouldUpdateLength
     * @param string|null   $connection
     */
    protected function updateColumnsInTable($table, $charset, $collation, $newLength = null, Closure $shouldUpdateLength = null, $connection = null)
    {
        $columnsToChange = [];

        foreach ($this->getColumnsFromTable($table, $connection) as $column) {
            $column = $this->extractInformationFromColumn($column);

            if ($this->isStringType($column)) {
                $sql = "CHANGE `%field%` `%field%` %type%%brackets% CHARACTER SET %charset% COLLATE %collation% %null% %default%";
                $search = ['%field%', '%type%', '%brackets%', '%charset%', '%collation%', '%null%', '%default%'];
                $replace = [
                    $column['field'],
                    $column['type'],
                    $column['type_brackets'] ? '(' . $column['type_brackets'] . ')' : '',
                    $charset,
                    $collation,
                    $column['null'] ? 'NULL' : 'NOT NULL',
                    is_null($column['default']) ? ($column['null'] ? 'DEFAULT NULL' : '') : 'DEFAULT \'' . $column['default'] . '\''
                ];

                if ($this->isStringTypeWithLength($column) && $shouldUpdateLength($column) && is_int($newLength) && $newLength > 0) {
                    $replace[2] = '(' . $newLength . ')';
                }

                $columnsToChange[] = trim(str_replace($search, $replace, $sql));
            }
        }

        if (count($columnsToChange) > 0) {
            $query = "ALTER TABLE `{$table}` " . implode(', ', $columnsToChange);

            $this->getDatabaseConnection($connection)->update($query);
        }
    }

    /**
     * Get a list of all the columns for the provided table. Returns an array of stdClass objects.
     *
     * @param string      $table
     * @param string|null $connection
     *
     * @return array
     */
    protected function getColumnsFromTable($table, $connection = null)
    {
        return $this->getDatabaseConnection($connection)->select('SHOW FULL COLUMNS FROM ' . $table);
    }

    /**
     * Convert a table's character set and collation.
     *
     * @param string      $table
     * @param string      $charset
     * @param string      $collation
     * @param string|null $connection
     */
    protected function convertTableCharacterSetAndCollation($table, $charset, $collation, $connection = null)
    {
        $query = "ALTER TABLE {$table} CONVERT TO CHARACTER SET {$charset} COLLATE {$collation}";
        $this->getDatabaseConnection($connection)->update($query);

        $query = "ALTER TABLE {$table} DEFAULT CHARACTER SET {$charset} COLLATE {$collation}";
        $this->getDatabaseConnection($connection)->update($query);
    }

    /**
     * Change the entire database's (The database represented by the connection) character set and collation.
     *
     * # Note: This must be done with the unprepared method, as PDO complains that the ALTER DATABASE command is not yet
     *         supported as a prepared statement.
     *
     * @param string      $charset
     * @param string      $collation
     * @param string|null $connection
     */
    protected function alterDatabaseCharacterSetAndCollation($charset, $collation, $connection = null)
    {
        $database = $this->getDatabaseConnection($connection)->getDatabaseName();

        $query = "ALTER DATABASE {$database} CHARACTER SET {$charset} COLLATE {$collation}";

        $this->getDatabaseConnection($connection)->unprepared($query);
    }
}

请在运行此之前请备份您的数据库。使用风险自负!

答案 1 :(得分:1)

数据库字符集和排序规则是新创建的表的默认设置是列的默认设置。

为每张桌子执行此操作:

ALTER TABLE table_name CONVERT TO utf8mb4;