如何将数学表达式拆分为标记？ （字符串-> [String]）

Question

我知道我的问题听起来很普通，我真的不知道如何用谷歌搜索这种事情，所以我将提供我的意思的示例：

ghci> splitTok "12 + 564 * (5 - 38) / 223"
["12", "+", "564", "*", "(", "5", "-", "38", ")", "/", "223"]
ghci> splitTok " 5+2-42  *    (46/5 )"
["5", "+", "2", "-", "42", "*", "(", "46", "/", "5", ")"]

如何实现这样的目标？也许Prelude中有一个函数可以执行这种操作？

Answer 1

似乎只是将数字分组在一起并删除空格。试试

import Data.Char (isSpace, isDigit)
import Data.List (groupBy)

splitTok = groupBy (\x y->isDigit x && isDigit y) . filter (not . isSpace)

Answer 2

好吧，我想出了一个可以做到这一点的丑陋的骇客。 （不幸的是引入了一个错误）

import Data.Char (isSpace, isDigit)

strip :: String -> String
strip = filter (not . isSpace)

addSpacing :: String -> String
addSpacing [a] = [a]
addSpacing (x:y:cs)
    | isDigit x && isDigit y = x : addSpacing rest
    | otherwise = x : ' ' : addSpacing rest
  where rest = y : cs

splitTok :: String -> [String]
splitTok = words . addSpacing . strip

在此示例中，它无法生成正确的令牌字符串：

ghci> splitTok "125 +   12 62 - 12  *( 51/  3)        "
["125","+","1262","-","12","*","(","51","/","3",")"]

对于大多数表达式，它仍然可以正常工作：

ghci> splitTok "4123-36522+12"
["4123","-","36522","+","12"]
ghci> splitTok "124 *(12 -(4+5*(331/7)))"
["124","*","(","12","-","(","4","+","5","*","(","331","/","7",")",")",")"]