冷漠地遍历数组或列表

时间:2018-09-17 20:41:05

标签: common-lisp sbcl

问题

假设您有许多列表或数组,为示例起见,假设两个:

(defparameter *arr* #(1 2 3))
(defparameter *list* '(4 5 6))

您可以使用loopacross关键字in覆盖它们:

(loop for elem across *arr* do (format t "~a" elem))
     => 123
(loop for elem in *list* do (format t "~a" elem))
     => 456

我希望能够使用相同的语法在这些数组或列表上loop。我正在使用SBCL,执行速度令人担忧。

使用being the elements of

此语法很好,因为无论其参数是list还是array,它都可以工作。

(loop for elem being the elements of *arr* do (format t "~a" elem))
     => 123
(loop for elem being the elements of *list* do (format t "~a" elem))
     => 456

但是它的速度太快了。如果我们通过访问100万次包含100个元素的列表或数组来进行快速比较:

(format t "# Test 1.1.1 : Accessing list of doubles with loop 'in': ") (terpri)
(let ((test-list (make-list 100 :initial-element 12.2d0))
      (testvar 0d0))
  (declare (optimize (speed 3))
           (type list test-list)
           (type double-float testvar))
  (time (dotimes (it 1000000 t) (loop for el in test-list do
                                      (setf testvar (the double-float el))))))

(format t "# Test 1.1.2 : Accessing list of doubles with loop 'elements': ") (terpri)
(let ((test-list (make-list 100 :initial-element 12.2d0))
      (testvar 0d0))
  (declare (optimize (speed 3))
           (type list test-list)
           (type double-float testvar))
  (time (dotimes (it 1000000 t) (loop for el being the elements of test-list do
                                      (setf testvar (the double-float el))))))

(format t "# Test 1.2.1 : Accessing simple-array of doubles using loop 'across' : ") (terpri)
(let ((test-array (make-array 100 :initial-element 12.2d0 :element-type 'double-float))
      (testvar 0d0))
  (declare (optimize (speed 3))
           (type double-float testvar)
           (type simple-array test-array))
  (time (dotimes (it 1000000 t) (loop for el across test-array do
                                      (setf testvar (the double-float el))))))

(format t "# Test 1.2.2 : Accessing simple-array of doubles using loop 'elements' : ") (terpri)
(let ((test-array (make-array 100 :initial-element 12.2d0 :element-type 'double-float))
      (testvar 0d0))
  (declare (optimize (speed 3))
           (type double-float testvar)
           (type simple-array test-array))
  (time (dotimes (it 1000000 t) (loop for el being the elements of test-array do
                                      (setf testvar (the double-float el))))))

它给我们:

# Test 1.1.1 : Accessing list of doubles with loop 'in': 
Evaluation took:
  0.124 seconds of real time
  0.123487 seconds of total run time (0.123471 user, 0.000016 system)
  99.19% CPU
  445,008,960 processor cycles
  672 bytes consed

# Test 1.1.2 : Accessing list of doubles with loop 'elements': 
Evaluation took:
  0.843 seconds of real time
  0.841639 seconds of total run time (0.841639 user, 0.000000 system)
  99.88% CPU
  3,034,104,192 processor cycles
  0 bytes consed

# Test 1.2.1 : Accessing simple-array of doubles using loop 'across' : 
Evaluation took:
  0.062 seconds of real time
  0.062384 seconds of total run time (0.062384 user, 0.000000 system)
  100.00% CPU
  224,896,032 processor cycles
  0 bytes consed

# Test 1.2.2 : Accessing simple-array of doubles using loop 'elements' : 
Evaluation took:
  1.555 seconds of real time
  1.554472 seconds of total run time (1.541572 user, 0.012900 system)
  [ Run times consist of 0.094 seconds GC time, and 1.461 seconds non-GC time. ]
  99.94% CPU
  5,598,161,100 processor cycles
  1,600,032,848 bytes consed

我认为它必须使用elt访问器?无论如何,速度上的损失是不可接受的。

尝试通过宏变得聪明

我写了一些东西,可以实现我对listarray具有相同语法的目标。我认为这不是很好,因为它似乎过于笨拙,但是在这里:

(defun forbuild (el-sym list-or-array list-or-array-sym)
  "Outputs either :
     * (for el-sym in list-or-array)
     * (for el-sym across list-or-array)
Depending on type of list-or-array.
el-sym : symbol, eg. 'it1
list-or-array : declared, actual data for list or array
list-or-array-sym : symbol name for the table, to avoid writing the data in full
                    in the 'loop' call using eval.
Example call : (forbuild 'it1 arr 'arr)"
  (cond ((typep list-or-array 'array)
         `(for ,el-sym across ,list-or-array-sym))
        ((typep list-or-array 'list)
         `(for ,el-sym in ,list-or-array-sym))))

(defun forbuild-l (l-elsyms l-lars l-larsyms)
  "forbuild but over lists of things."
  (let ((for-list nil)
        (list-temp nil))
    (loop for elem in l-elsyms
          for lar in l-lars
          for larsym in l-larsyms do
          (setf list-temp (forbuild elem lar larsym))
          (loop for word-temp in list-temp do
                (push word-temp for-list)))
    (nreverse for-list)))

(defun loop-expr (forlist body)
  "Creates the expression ready to be evaluated to execute the loop.
forlist : List of symbols to be inserted syntactically. eg.
          FOR IT1 ACROSS ARR1 FOR IT2 IN ARR2
body : all the expression after the 'for' clauses in the 'loop'."
  `(loop ,@forlist ,@body))

(defmacro looparl (element list-or-array &rest body)
  (let ((forlist (gensym)))
    `(let ((,forlist (forbuild2-l (quote ,element)
                                  (list ,@list-or-array)
                                  (quote ,list-or-array))))
       (loop-expr ,forlist (quote ,body)))))

基本上,我从参数构建正确的loop语法。此处给出的looparl的版本可以这样称呼:

(let ((arr1 #(7 8 9))
      (list2 (list 10 11 12)))
    (looparl (it1 it2) (arr1 list2) do (format t "~a ~a" it1 it2) (terpri)))
=> (LOOP FOR IT1 ACROSS ARR1
  FOR IT2 IN LIST2
  DO (FORMAT T "~a ~a" IT1 IT2) (TERPRI))

在此示例中,将忽略此输出表达式的实际求值,因为它不适用于非全局名称。如果我们在looparl的末尾插入一个eval:

(defmacro looparl (element list-or-array &rest body)
  (let ((forlist (gensym)))
    `(let ((,forlist (forbuild2-l (quote ,element)
                                  (list ,@list-or-array)
                                  (quote ,list-or-array))))
       (eval (loop-expr ,forlist (quote ,body))))))

处理全局变量,我们发现仍然存在速度问题,因为在运行时会进行评估:

(looparl (it1 it2) (*arr* *list*) for it from 100
              do (format t "~a ~a ~a" it1 it2 it) (terpri))
=> 1 4 100
   2 5 101
   3 6 102
(time (dotimes (iter 1000 t) (looparl (it1 it2) (*arr* *list*) for it from 100
              do (format t "~a ~a ~a" it1 it2 it) (terpri))))
=> Evaluation took:
  1.971 seconds of real time
  1.932610 seconds of total run time (1.892329 user, 0.040281 system)
  [ Run times consist of 0.097 seconds GC time, and 1.836 seconds non-GC time. ]
  98.07% CPU
  1,000 forms interpreted
  16,000 lambdas converted
  7,096,353,696 processor cycles
  796,545,680 bytes consed

每个宏一次评估一千次。但是可以确定类型在编译时是否已知? looparl中的语法类型非常好,我希望能够在不影响速度的情况下使用它。

我在Peter Seibel's book Practical Common Lisp, chapter "Loop for Black Belts"

中阅读了此笔记
  

3您可能想知道为什么LOOP在不需要其他介词的情况下无法弄清是在列表还是向量上循环。这是LOOP成为宏的另一个结果:列表或向量的值要到运行时才能知道,但是LOOP作为宏必须在编译时生成代码。而且LOOP的设计师希望它生成非常有效的代码。为了能够生成有效的代码来遍历一个向量,它需要在编译时知道该值在运行时将是一个向量-因此,需要不同的介词。

我犯了一些常见的Lisp废话吗?您将如何创建一个快速运行的looparl

编辑1:FOR

非常感谢您引用FOR libraryover函数中的for:for关键字确实正是我所需要的。但是,基准测试确实令人难以置信:

(let ((test-list (make-list 100 :initial-element 12.2d0))
      (testvar 0d0))
  (declare (optimize (speed 3))
           (type list test-list)
           (type double-float testvar))
  (time (dotimes (it 1000000 t) 
          (for:for ((el over test-list))
            (setf testvar (the double-float el))))))

(let ((test-array (make-array 100 :initial-element 12.2d0))
      (testvar 0d0))
  (declare (optimize (speed 3))
           (type simple-array test-array)
           (type double-float testvar))
  (time (dotimes (it 1000000 t) 
          (for:for ((el over test-array))
            (setf testvar (the double-float el))))))

Evaluation took:
  4.802 seconds of real time
  4.794485 seconds of total run time (4.792492 user, 0.001993 system)
  [ Run times consist of 0.010 seconds GC time, and 4.785 seconds non-GC time. ]
  99.83% CPU
  17,286,934,536 processor cycles
  112,017,328 bytes consed

Evaluation took:
  6.758 seconds of real time
  6.747879 seconds of total run time (6.747879 user, 0.000000 system)
  [ Run times consist of 0.004 seconds GC time, and 6.744 seconds non-GC time. ]
  99.85% CPU
  24,329,311,848 processor cycles
  63,995,808 bytes consed

使用专用关键字inacross的该库的速度与标准loop相同。但是使用over的速度非常慢。

编辑2:mapetypecase

感谢sds和Rainer Joswig的建议。实际上,对于只有一个数组/列表进行迭代的简单情况,它确实可以工作。让我告诉您一个我想到的用例:我正在实现一个gnuplot包装器,既作为培训,又在工具箱中拥有自己的程序。我想以不同的方式从用户列表或数组中获取内容,以作为序列传递给gnuplot。这就是为什么我需要能够同时遍历多个数组/列表,以及使用用于迭代编号等的优雅循环子句。

在这个用例(gnuplot包装器)中,我的for中每个“数据块”只有两个或三个loop子句,因此我考虑过根据类型编写每种组合手动输入是可能的,但是很尴尬。如果我不得不做类似的事情,我会被困住:

(loop for el1 in list1
      for el2 across arr1
      for el3 in list2
      for el4 in list3
      ...)

输入list-iarr-i。此用例的另一个备用计划是将所有内容都转换为数组。

我认为,由于它很容易被概念化,因此我可以一劳永逸地编写出灵活而又快速的内容,但是一定有理由说明它既不在规范中,也不在SBCL专用代码中。

4 个答案:

答案 0 :(得分:3)

对于琐碎的用途,您可能会这么做

(flet ((do-something (e)
         ...))
  (etypecase foo
    (vector (loop for e across foo do (do-something e)))
    (list   (loop for e in     foo do (do-something e))))

运行时类型分配可能比使用序列抽象的通用迭代构造更快。

答案 1 :(得分:2)

您正在寻找的被称为 map: 两者

(map nil #'princ '(1 2 3))

(map nil #'princ #(1 2 3))

打印123

但是,列表和数组是非常不同的野兽,最好事先确定要使用哪一个。

答案 2 :(得分:2)

Shimmera的For库具有通用的over迭代器:

(ql:quickload "for")

(for:for ((a over *arr*)
          (b over *list*))
       (print (list a b)))

;; (1 4) 
;; (2 5) 
;; (3 6) 

它还具有“ in”和“ accross”,因此在开发过程中使用“ over”并在以后进行细化可能会有所帮助。

我让你做基准测试:)

答案 3 :(得分:2)

将数组强制为列表,然后循环执行,其性能与开始时的列表相同,虽然不如array好,但不如使用element差,它确实可以无需使用其他机制即可使用列表或数组:

(loop for x in (coerce array 'list) do something)