我正在尝试将文本片段与固定数量的列对齐。该文本用于记录目的,可能包含一些用户数据,因此我不能假设任何有关输入的内容。为了便于查看,我想确保在标准LINUX控制台中查看时,此文本只有一定数量的固定字符。
PHP没有与其wordwrap - 函数等效的多字节。有几个框架有自己的版本。我一直在尝试几个多字节函数,其中包括this question中的答案,以便为unicode显示编码一些UTF-8文本。大多数答案似乎使用'mb_strlen'来计算子串长度。
为了确保我想要显示的文件被识别为UTF-8,我在文本前面添加了一个字节顺序标记。据我所知,这'应该'导致linux正确识别和格式化。
然后,我尝试编码一些长字符串。在指定的N个字符位置之后,诸如“自上车......”之类的字符串将被正确切断。但是,诸如“₩箕式紫甘氨檀紫红色₩......”之类的字符串则不会。它包含一些在UNIX控制台中具有双倍长度的字符,并且会比预期更晚切断。事实上,在这种情况下,它似乎只是预期显示的字符数的两倍,因此看起来部分或全部双宽字符被读取为宽度1.常量'mb_internal_encoding'正确设置为UTF-8,使用:
mb_internal_encoding("UTF-8");
请注意,当我实际将mb_internal_encoding设置为其他值时,我得到其他奇怪的结果,然后字符串将被截止50%太快,因此在位置26处切断70个字符的字符串。 -offs发生在角色的中间,导致一些mojibake。
我执行工作的类中的换行符代码如下所示,mb_wordwrap重定向到其中一个示例函数:
private function BreakIntoLines($text) {
$dtxt = $this->mb_wordwrap($text, self::LINE_LENGTH, PHP_EOL, true);
return explode(PHP_EOL, $dtxt);
}
然后使用经典的c风格文件编写函数进行文件编写,下面的代码片段会编写其中一行:
fwrite($this->file, $line);
fwrite($this->file, PHP_EOL);
我使用一系列UTF字符验证了输出,它们似乎都正确显示。我似乎完全迷失了问题,这里发生了什么?
注意:尝试自己编写算法似乎是一项过度的工作。似乎也有组合字符。例如,序列\ u68 \ u0300实际上只占用终端中的一个字符。试验,代码:
$str = json_decode("\"\\u0068\\u0300\"");
var_dump($str);
echo mb_strlen($str);
打印出:string(3)“h”2
答案 0 :(得分:2)
对于这个问题的一部分,有一个很好的和干净的解决方案。使用链接问题中的mb_wordwrap函数,但将mb_*
函数替换为grapheme_*
函数。字素是' intl
'的一部分。 php的扩展名,实际上可以正确计算字符长度。有关详细信息,另请参阅this page。
因此,我们使用grapheme_strlen
代替mb_strlen
,grapheme_substr
代替mb_substr
,依此类推。然后在自动换行函数中,我们可以将字符串视为字形集合,这些字形是内部C风格的php字符串,每个字符串都带有N个ASCII符号。这实际上是mb_ *函数运行的方式,因此代码保留了基本相同的结构。
例如:
$str = json_decode("\"\\u0068\\u0300\"");
var_dump($str);
echo grapheme_strlen($str);
将输出:string(3) "h̀" 1
。
它仍然无法一次性正确计算更宽的字符。因此,为了实际完全解决问题,我们需要一个实际的实现。有一个C实现使用了双宽字符的范围列表和一个带有二进制遍历的更长列表,用于所有0宽字符,这在php中重新实现。
下面的代码理论上应该正确地包装正常文本,同时还支持所有奇怪的UTF-8字符。它不支持带有其他包装规则的语言(包装在非空格字符中),也可能不完全支持奇怪的空格(ASCII范围之外的空格)。但是,此代码应保证$width
个字符的行宽。
/**
* Word-wrap a multi-byte character (UTF) string
* @param string $string Initial string.
* @param int $width Maximum width of a line
* @param string $break Which character(s) to line break with
* @param bool $cut Whether to force-chop long words.
* @return string The chopped string.
*/
private function mb_wordwrap($string, $width = 75, $break = "\n", $cut = false) {
$string = (string) $string;
if ($string === '') {
return '';
}
$break = (string) $break;
if ($break === '') {
trigger_error('Break string cannot be empty', E_USER_ERROR);
}
$width = (int) $width;
if ($width === 0 && $cut) {
trigger_error('Cannot force cut when width is zero', E_USER_ERROR);
}
if (mb_check_encoding($string, 'ASCII')) {
return wordwrap($string, $width, $break, $cut);
}
$result = '';
// Width on display
// note: stringLength != stringWidth!
$breakWidth = $this->trueStringWidth($string);
$lastStartWidth = $lastSpaceWidth = 0;
// These all measure 'length'.
// Length in characters
$breakLength = strlen($break);
$lastStartLength = $lastSpaceLength = 0;
$G_sz = grapheme_strlen($string);
$wPos = 0;
$lPos = 0;
// Iterate over graphemes
// Measure using TrueWidth
// Cut using ASCII (for speed)
for($i = 0; $i < $G_sz; ++$i) {
$char = grapheme_substr($string, $i, 1);
$charLength = strlen($char);
$charWidth = $this->trueStringWidth($char);
$lookahead_wPos = $wPos + $charWidth;
// If we already have a line break, preserve it and start anew
if($breakLength !== 1) {
$possibleBreak = substr($string, $lPos, $breakLength);
} else {
$possibleBreak = substr($string, $lPos, $breakLength);
}
if ($possibleBreak === $break) {
$result .= substr($string, $lastStartLength, $lPos - $lastStartLength + $breakLength);
$lPos += $breakLength - $charLength;
$wPos += $breakWidth - $charWidth;
$lastStartLength = $lastSpaceLength = $breakLength;
$lastStartWidth = $lastSpaceWidth = $breakWidth;
continue;
}
// if we match any 'whitespace' character,
if(preg_match("/\\h/u", $char)) {
// Exclude the space itself, do not use the lookahead
if($wPos - $lastStartWidth >= $width) {
$result .= substr($string, $lastStartLength, $lPos - $lastStartLength) . $break;
$lastStartLength = $lPos + $charLength;
$lastStartWidth = $wPos + $charWidth;
}
$lastSpaceWidth = $wPos;
$lastSpaceLength = $lPos;
continue;
}
// look-ahead one character
$nextChar = grapheme_substr($string, $i+1, 1);
// If we are about to overflow, AND the last space was too far back,
if($cut && $lookahead_wPos - $lastStartWidth > $width && $lastStartWidth >= $lastSpaceWidth) {
$result .= substr($string, $lastStartLength, $lPos - $lastStartLength) . $break;
$lastStartLength = $lPos;
$lastStartWidth = $wPos;
continue;
}
if ($lookahead_wPos - $lastStartWidth > $width && $lastStartWidth < $lastSpaceWidth) {
$result .= substr($string, $lastStartLength, $lastSpaceLength - $lastStartLength) . $break;
$lastStartLength = $lastSpaceLength = $lastSpaceLength + $charLength;
$lastStartWidth = $lastSpaceWidth = $lastSpaceWidth + $charWidth;
continue;
}
$wPos += $charWidth;
$lPos += $charLength;
}
if($lastStartLength !== $lPos) {
$result .= substr($string, $lastStartLength, $lPos - $lastStartLength);
}
return $result;
}
private function trueStringWidth($str) {
$w = 0;
for($i = 0; $i < mb_strlen($str); ++$i) {
$char = mb_substr($str, $i, 1);
$w += $this->trueCharWidth($char);
}
return $w;
}
private function trueCharWidth($char) {
$ucs = $this->uniord($char);
// For non-unicode characters, return 1.
// Consoles replace them with 'replacement characters' which have width 1!
if($ucs === FALSE) {return 1;}
// Do some bit math...
$combi = [
[ 0x0300, 0x036F ], [ 0x0483, 0x0486 ], [ 0x0488, 0x0489 ],
[ 0x0591, 0x05BD ], [ 0x05BF, 0x05BF ], [ 0x05C1, 0x05C2 ],
[ 0x05C4, 0x05C5 ], [ 0x05C7, 0x05C7 ], [ 0x0600, 0x0603 ],
[ 0x0610, 0x0615 ], [ 0x064B, 0x065E ], [ 0x0670, 0x0670 ],
[ 0x06D6, 0x06E4 ], [ 0x06E7, 0x06E8 ], [ 0x06EA, 0x06ED ],
[ 0x070F, 0x070F ], [ 0x0711, 0x0711 ], [ 0x0730, 0x074A ],
[ 0x07A6, 0x07B0 ], [ 0x07EB, 0x07F3 ], [ 0x0901, 0x0902 ],
[ 0x093C, 0x093C ], [ 0x0941, 0x0948 ], [ 0x094D, 0x094D ],
[ 0x0951, 0x0954 ], [ 0x0962, 0x0963 ], [ 0x0981, 0x0981 ],
[ 0x09BC, 0x09BC ], [ 0x09C1, 0x09C4 ], [ 0x09CD, 0x09CD ],
[ 0x09E2, 0x09E3 ], [ 0x0A01, 0x0A02 ], [ 0x0A3C, 0x0A3C ],
[ 0x0A41, 0x0A42 ], [ 0x0A47, 0x0A48 ], [ 0x0A4B, 0x0A4D ],
[ 0x0A70, 0x0A71 ], [ 0x0A81, 0x0A82 ], [ 0x0ABC, 0x0ABC ],
[ 0x0AC1, 0x0AC5 ], [ 0x0AC7, 0x0AC8 ], [ 0x0ACD, 0x0ACD ],
[ 0x0AE2, 0x0AE3 ], [ 0x0B01, 0x0B01 ], [ 0x0B3C, 0x0B3C ],
[ 0x0B3F, 0x0B3F ], [ 0x0B41, 0x0B43 ], [ 0x0B4D, 0x0B4D ],
[ 0x0B56, 0x0B56 ], [ 0x0B82, 0x0B82 ], [ 0x0BC0, 0x0BC0 ],
[ 0x0BCD, 0x0BCD ], [ 0x0C3E, 0x0C40 ], [ 0x0C46, 0x0C48 ],
[ 0x0C4A, 0x0C4D ], [ 0x0C55, 0x0C56 ], [ 0x0CBC, 0x0CBC ],
[ 0x0CBF, 0x0CBF ], [ 0x0CC6, 0x0CC6 ], [ 0x0CCC, 0x0CCD ],
[ 0x0CE2, 0x0CE3 ], [ 0x0D41, 0x0D43 ], [ 0x0D4D, 0x0D4D ],
[ 0x0DCA, 0x0DCA ], [ 0x0DD2, 0x0DD4 ], [ 0x0DD6, 0x0DD6 ],
[ 0x0E31, 0x0E31 ], [ 0x0E34, 0x0E3A ], [ 0x0E47, 0x0E4E ],
[ 0x0EB1, 0x0EB1 ], [ 0x0EB4, 0x0EB9 ], [ 0x0EBB, 0x0EBC ],
[ 0x0EC8, 0x0ECD ], [ 0x0F18, 0x0F19 ], [ 0x0F35, 0x0F35 ],
[ 0x0F37, 0x0F37 ], [ 0x0F39, 0x0F39 ], [ 0x0F71, 0x0F7E ],
[ 0x0F80, 0x0F84 ], [ 0x0F86, 0x0F87 ], [ 0x0F90, 0x0F97 ],
[ 0x0F99, 0x0FBC ], [ 0x0FC6, 0x0FC6 ], [ 0x102D, 0x1030 ],
[ 0x1032, 0x1032 ], [ 0x1036, 0x1037 ], [ 0x1039, 0x1039 ],
[ 0x1058, 0x1059 ], [ 0x1160, 0x11FF ], [ 0x135F, 0x135F ],
[ 0x1712, 0x1714 ], [ 0x1732, 0x1734 ], [ 0x1752, 0x1753 ],
[ 0x1772, 0x1773 ], [ 0x17B4, 0x17B5 ], [ 0x17B7, 0x17BD ],
[ 0x17C6, 0x17C6 ], [ 0x17C9, 0x17D3 ], [ 0x17DD, 0x17DD ],
[ 0x180B, 0x180D ], [ 0x18A9, 0x18A9 ], [ 0x1920, 0x1922 ],
[ 0x1927, 0x1928 ], [ 0x1932, 0x1932 ], [ 0x1939, 0x193B ],
[ 0x1A17, 0x1A18 ], [ 0x1B00, 0x1B03 ], [ 0x1B34, 0x1B34 ],
[ 0x1B36, 0x1B3A ], [ 0x1B3C, 0x1B3C ], [ 0x1B42, 0x1B42 ],
[ 0x1B6B, 0x1B73 ], [ 0x1DC0, 0x1DCA ], [ 0x1DFE, 0x1DFF ],
[ 0x200B, 0x200F ], [ 0x202A, 0x202E ], [ 0x2060, 0x2063 ],
[ 0x206A, 0x206F ], [ 0x20D0, 0x20EF ], [ 0x302A, 0x302F ],
[ 0x3099, 0x309A ], [ 0xA806, 0xA806 ], [ 0xA80B, 0xA80B ],
[ 0xA825, 0xA826 ], [ 0xFB1E, 0xFB1E ], [ 0xFE00, 0xFE0F ],
[ 0xFE20, 0xFE23 ], [ 0xFEFF, 0xFEFF ], [ 0xFFF9, 0xFFFB ],
[ 0x10A01, 0x10A03 ], [ 0x10A05, 0x10A06 ], [ 0x10A0C, 0x10A0F ],
[ 0x10A38, 0x10A3A ], [ 0x10A3F, 0x10A3F ], [ 0x1D167, 0x1D169 ],
[ 0x1D173, 0x1D182 ], [ 0x1D185, 0x1D18B ], [ 0x1D1AA, 0x1D1AD ],
[ 0x1D242, 0x1D244 ], [ 0xE0001, 0xE0001 ], [ 0xE0020, 0xE007F ],
[ 0xE0100, 0xE01EF ]
];
/* test for 8-bit control characters */
if ($ucs === 0)
return 0;
if ($ucs < 32 || ($ucs >= 0x7f && $ucs < 0xa0))
return 0;
/* binary search in table of non-spacing characters */
if ($this->binaryIntervalSearch($combi, $ucs))
return 0;
/* if we arrive here, ucs is not a combining or C0/C1 control character */
return 1 +
($ucs >= 0x1100 &&
($ucs <= 0x115f || /* Hangul Jamo init. consonants */
$ucs == 0x2329 || $ucs == 0x232a ||
($ucs >= 0x2e80 && $ucs <= 0xa4cf &&
$ucs != 0x303f) || /* CJK ... Yi */
($ucs >= 0xac00 && $ucs <= 0xd7a3) || /* Hangul Syllables */
($ucs >= 0xf900 && $ucs <= 0xfaff) || /* CJK Compatibility Ideographs */
($ucs >= 0xfe10 && $ucs <= 0xfe19) || /* Vertical forms */
($ucs >= 0xfe30 && $ucs <= 0xfe6f) || /* CJK Compatibility Forms */
($ucs >= 0xff00 && $ucs <= 0xff60) || /* Fullwidth Forms */
($ucs >= 0xffe0 && $ucs <= 0xffe6) ||
($ucs >= 0x20000 && $ucs <= 0x2fffd) ||
($ucs >= 0x30000 && $ucs <= 0x3fffd)));
}
private function uniord($c) {
if (ord($c{0}) >=0 && ord($c{0}) <= 127) {
return ord($c{0});
}
if (ord($c{0}) >= 192 && ord($c{0}) <= 223) {
return (ord($c{0})-192)*64 + (ord($c{1})-128);
}
if (ord($c{0}) >= 224 && ord($c{0}) <= 239) {
return (ord($c{0})-224)*4096 + (ord($c{1})-128)*64 + (ord($c{2})-128);
}
if (ord($c{0}) >= 240 && ord($c{0}) <= 247) {
return (ord($c{0})-240)*262144 + (ord($c{1})-128)*4096 + (ord($c{2})-128)*64
+ (ord($c{3})-128);
}
if (ord($c{0}) >= 248 && ord($c{0}) <= 251) {
return (ord($c{0})-248)*16777216 + (ord($c{1})-128)*262144 + (ord($c{2})-
128)*4096 + (ord($c{3})-128)*64 + (ord($c{4})-128);
}
if (ord($c{0}) >= 252 && ord($c{0}) <= 253) {
return (ord($c{0})-252)*1073741824 + (ord($c{1})-128)*16777216 + (ord($c{2})-
128)*262144 + (ord($c{3})-128)*4096 + (ord($c{4})-128)*64 +
(ord($c{5})-128);
}
if (ord($c{0}) >= 254 && ord($c{0}) <= 255) { // error
return FALSE;
}
return 0;
} // function _uniord()
// It is assumed the interval array is sorted!
// It is assumed we have a SIMPLE array (indexed 0, 1, 2, ...).
private function binaryIntervalSearch($array, $element) {
if(count($array) === 1) {
if($array[0][0] <= $element && $element <= $array[0][1]) {
return true;
} else {
return false;
}
} else if(count($array) === 0) {
return false;
}
// split the array into two halves and a central element.
$tC = count($array) >> 1;
// rightmost left element
if($array[$tC-1][1] >= $element) {
return $this->binaryIntervalSearch(array_slice($array, 0, $tC), $element);
} else if($array[$tC][0] <= $element) {
return $this->binaryIntervalSearch(array_slice($array, $tC), $element);
}
return false;
}