Question

我一直承认我对镜头不好，但不是通过例子学习是件好事吗？我想使用HTML，使用taggy-lens解析它，然后从内部删除所有script元素。这是我的尝试：

#!/usr/bin/env stack
-- stack --resolver lts-7.1 --install-ghc runghc --package text --package lens --package taggy-lens --package string-class --package classy-prelude

{-# LANGUAGE NoImplicitPrelude #-}
{-# LANGUAGE OverloadedStrings #-}

import ClassyPrelude
import Control.Lens hiding (children, element)
import Data.String.Class (toText, fromText, toString)
import Data.Text (Text)
import Text.Taggy.Lens
import qualified Text.Taggy.Lens as Taggy
import qualified Text.Taggy.Renderer as Renderer

somehtmlSmall :: Text
somehtmlSmall =
    "<!doctype html><html><body>\
    \<div id=\"article\"><div>first</div><div>second</div><script>this should be removed</script><div>third</div></div>\
    \</body></html>"

renderWithoutScriptTag :: Text
renderWithoutScriptTag =
    let mArticle :: Maybe Taggy.Element
        mArticle =
            (fromText somehtmlSmall) ^? html .
            allAttributed (ix "id" . only "article")
        mArticleFiltered =
            fmap
                (\el ->
                      el ^.. to universe . traverse .
                      filtered (\n -> n ^. name /= "script"))
                mArticle
    in maybe "" (toText . concatMap Renderer.render) mArticleFiltered

main :: IO ()
main = print renderWithoutScriptTag

将此文件标记为可执行文件并运行它，您将看到：

➜  tmp  ./scraping-question.hs
"<div id=\"article\"><div>first</div><div>second</div><script>this should be removed</script><div>third</div></div><div>first</div><div>second</div><div>third</div>"

所以，这没用。我想：

有一个有效的解决方案
了解工作解决方案

如果你能帮我意识到我的错误，那将特别感激。谢谢！

Answer 1

问题的根源是universe，它将DOM树展平为一个列表。如果你再看一下输出，你会看到过滤工作正常，但树结构丢失了 - 所以你得到了未经修改的文章元素（所有孩子都在其中），然后是子节点减去脚本元素。 / p>

一个可以做你想做的Control.Lens.Plated组合器是transform，它以自下而上的方式转换“树中的每个元素”：

transform :: Plated a => (a -> a) -> a -> a

特别是，您可以使用它以递归方式过滤子节点：

renderWithoutScriptTag :: Text
renderWithoutScriptTag =
    let mArticle :: Maybe Taggy.Element
        mArticle =
            (fromText somehtmlSmall) ^? html .
            allAttributed (ix "id" . only "article")
        mArticleFiltered =
            fmap
                (transform (children %~ filter (\n ->
                    n ^? element . name /= Just "script")))
                mArticle
    in maybe "" (toText . Renderer.render) mArticleFiltered

通过镜头

1 个答案: