Question

我正在尝试通过assignment 1获得斯坦福大学cs244n课程。问题1b强烈建议优化Softmax功能。我设法得到了N维向量的Softmax。我还得到了MxN维矩阵的Softmax，但是在列中使用了for循环。我有以下代码：

def softmax(x):
    orig_shape = x.shape

    # Matrix
    if len(x.shape) > 1:
        softmax = np.zeros(orig_shape)
        for i,col in enumerate(x):
            softmax[i] = np.exp(col - np.max(col))/np.sum(np.exp(col - np.max(col)))
    # Vector
    else:
        softmax = np.exp(x - np.max(x))/np.sum(np.exp(x - np.max(x)))
    return softmax

我可以实施更优化的Matrix实现吗？

Answer 1

在相关NumPy broadcasting上使用ufuncs并涵盖通用维数的ndarray -

public class ExampleWriter {

    public static final int COUNT = 10_000;
    public static final String FILE = "test.csv";

    public static void main(String[] args) throws Exception {
        try (OutputStream os = new FileOutputStream(FILE)){         
            os.write(239);
            os.write(187);
            os.write(191);
            BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(os, StandardCharsets.UTF_8));         
            for (int i = 0; i < COUNT; i++) {               
                writer.write(Integer.toString(i));
                writer.newLine();               
            }           
        } catch (IOException e) {                       
            e.printStackTrace();
        }

        System.out.println(checkLineCount(COUNT, new File(FILE)));
    }

    public static String checkLineCount(int expectedLineCount, File file) throws Exception {
        BufferedReader expectedReader = new BufferedReader(new FileReader(file));
        try {
            int lineCount = 0;
            while (expectedReader.readLine() != null) {
                lineCount++;
            }
            if (expectedLineCount == lineCount) {
                return "correct";       
            } else {
                return "incorrect"; 
            }
        }
        finally {
            expectedReader.close();
        }
    }
}

Answer 2

您可以尝试使用np.apply_along_axis，您必须指定执行代码的轴（在您的情况下为axis=1）。这是一个有效的例子：

In [1]: import numpy as np

In [2]: def softmax(x):
   ...:     orig_shape = x.shape
    ...: 
   ...:     # Matrix
   ...:     if len(x.shape) > 1:
   ...:         softmax = np.zeros(orig_shape)
   ...:         for i,col in enumerate(x):
   ...:             softmax[i] = np.exp(col - np.max(col))/np.sum(np.exp(col - np.max(col)))
   ...:     # Vector
   ...:     else:
   ...:         softmax = np.exp(x - np.max(x))/np.sum(np.exp(x - np.max(x)))
   ...:     return softmax
   ...: 

In [3]: def softmax_vectorize(x):
   ...:     return np.exp(x - np.max(x))/np.sum(np.exp(x - np.max(x)))
   ...: 

In [4]: X = np.array([[1, 0, 0, 4, 5, 0, 7],
   ...:            [1, 0, 0, 4, 5, 0, 7],
   ...:            [1, 0, 0, 4, 5, 0, 7]])

In [5]: print softmax(X)
[[  2.08239574e-03   7.66070581e-04   7.66070581e-04   4.18260365e-02
    1.13694955e-01   7.66070581e-04   8.40098401e-01]
 [  2.08239574e-03   7.66070581e-04   7.66070581e-04   4.18260365e-02
    1.13694955e-01   7.66070581e-04   8.40098401e-01]
 [  2.08239574e-03   7.66070581e-04   7.66070581e-04   4.18260365e-02
    1.13694955e-01   7.66070581e-04   8.40098401e-01]]

In [6]: print np.apply_along_axis(softmax_vecorize, axis=1, arr=X)
[[  2.08239574e-03   7.66070581e-04   7.66070581e-04   4.18260365e-02
    1.13694955e-01   7.66070581e-04   8.40098401e-01]
 [  2.08239574e-03   7.66070581e-04   7.66070581e-04   4.18260365e-02
    1.13694955e-01   7.66070581e-04   8.40098401e-01]
 [  2.08239574e-03   7.66070581e-04   7.66070581e-04   4.18260365e-02
    1.13694955e-01   7.66070581e-04   8.40098401e-01]]

如何对多维矩阵的Softmax概率进行向量化

2 个答案: