Catboost提供dynamic C library，理论上可以使用任何编程语言。
我正在尝试使用JNA通过Java调用它。
我遇到CalcModelPrediction函数的问题，在header file中定义如下：
EXPORT bool CalcModelPrediction(
ModelCalcerHandle* calcer,
size_t docCount,
const float** floatFeatures, size_t floatFeaturesSize,
const char*** catFeatures, size_t catFeaturesSize,
double* result, size_t resultSize);
在Java中，我已经定义了接口函数如下：
public interface CatboostModel extends Library {
public Pointer ModelCalcerCreate();
public String GetErrorString();
public boolean LoadFullModelFromFile(Pointer calcer, String filename);
public boolean CalcModelPrediction(Pointer calcer, int docCount,
PointerByReference floatFeatures, int floatFeaturesSize,
PointerByReference catFeatures, int catFeaturesSize,
Pointer result, int resultSize);
public int GetFloatFeaturesCount(Pointer calcer);
public int GetCatFeaturesCount(Pointer calcer);
}
然后我这样称呼它：
CatboostModel catboost;
Pointer modelHandle;
catboost = Native.loadLibrary("catboostmodel", CatboostModel.class);
modelHandle = catboost.ModelCalcerCreate();
if (!catboost.LoadFullModelFromFile(modelHandle, "catboost_test.model"))
{
throw new RuntimeException("Cannot load Catboost model.");
}
final PointerByReference ppFloatFeatures = new PointerByReference();
final PointerByReference ppCatFeatures = new PointerByReference();
final Pointer pResult = new Memory(Native.getNativeSize(Double.TYPE));
float[] floatFeatures = {0.5f, 0.8f, 0.3f, 0.3f, 0.1f, 0.5f, 0.4f, 0.8f, 0.3f, 0.3f} ;
String[] catFeatures = {"1", "2", "3", "4"};
int catFeaturesLength = 0;
for (String s : catFeatures)
{
catFeaturesLength += s.length() + 1;
}
try
{
final Pointer pFloatFeatures = new Memory(floatFeatures.length * Native.getNativeSize(Float.TYPE));
for (int dloop=0; dloop<floatFeatures.length; dloop++) {
pFloatFeatures.setFloat(dloop * Native.getNativeSize(Float.TYPE), floatFeatures[dloop]);
}
ppFloatFeatures.setValue(pFloatFeatures);
final Pointer pCatFeatures = new Memory(catFeaturesLength * Native.getNativeSize(Character.TYPE));
long offset = 0;
for (final String s : catFeatures) {
pCatFeatures.setString(offset, s);
pCatFeatures.setMemory(offset + s.length(), 1, (byte)(0));
offset += s.length() + 1;
}
ppCatFeatures.setValue(pCatFeatures);
}
catch (Exception e)
{
throw new RuntimeException("Couldn't initialize parameters for catboost");
}
try
{
if (!catboost.CalcModelPrediction(
modelHandle,
1,
ppFloatFeatures, 10,
ppCatFeatures, 4,
pResult, 1
))
{
throw new RuntimeException("No prediction made: " + catboost.GetErrorString());
}
else
{
double[] result = pResult.getDoubleArray(0, 1);
log.info("Catboost prediction: " + String.valueOf(result[0]));
Assert.assertFalse("ERROR: Result empty", result.length == 0);
}
}
catch (Exception e)
{
throw new RuntimeException("Prediction failed: " + e);
}
我尝试将Pointer，PointerByReference和Pointer[]传递给CalcModelPrediction函数代替float **floatFeatures和char ***catFeatures，但没有任何效果。我总是会遇到分段错误，大概是CalcModelPrediction函数通过调用floatFeatures和catFeatures来尝试获取floatFeatures[0][0]和catFeatures[0][0]的元素时。
所以问题是，将多维数组从Java传递到JNA到C的正确方法是什么，它可以被视为指向值的指针？
有趣的是CalcModelPredictionFlat函数仅接受float **floatFeatures，然后只调用*floatFeatures，在传递PointerByReference时效果非常好。
更新 - 5.5.2018
第1部分
通过稍微修改原始Catboost .cpp和.h文件并重新编译libcatboost.so库来尝试调试segfault之后，我发现segfault是由于我将C中的size_t映射到{{ Java中的1}}修复此问题后，我在Java中的接口函数如下所示：
int public interface CatboostModel extends Library {
public boolean LoadFullModelFromFile(Pointer calcer, String filename);
public boolean CalcModelPrediction(Pointer calcer, size_t docCount,
Pointer[] floatFeatures, size_t floatFeaturesSize,
String[] catFeatures, size_t catFeaturesSize,
Pointer result, size_t resultSize);
}
类定义如下：
size_t 第2部分
进一步了解Catboost代码，我注意到行public static class size_t extends IntegerType {
public size_t() { this(0); }
public size_t(long value) { super(Native.SIZE_T_SIZE, value); }
}
正在访问**floatFeatures，floatFeatures[i]可以访问***catFeatures，catFetures[i][catFeatureIdx]可以访问floatFeatures。{ / p>
将Java中的Pointer更改为catFeatures数组后，我的代码开始使用没有分类功能的模型，即catFeatures长度为零。
然而，这个技巧不适用于通过双下标运算符[i][catFeatureidx]访问的char **catFeatures。所以现在，我修改了原始的Catboost代码，以便它接受String[] catFeatures - 一个字符串数组。在Java接口函数中，我设置了import requests
my_url = requests.get('http://www.vivoelfutbol.com.mx/futbolmexicano.php')
from bs4 import BeautifulSoup as soup
page_soup = soup(my_url.text, "html.parser")
containers = page_soup.findAll("div", {"class":"det"})
for result in containers:
time = result.find('div', {'class':'hor'}).text
hometeam = result.find('a').text
score = result.find('a', {'title':'Analisis y Antecedentes'}).text
awayteam = result.find('div', {'class':'eqvc'}).text
print(time + " " + hometeam + " " + score + " " + awayteam)
。现在我可以一次预测一个元素，这是不理想的。
答案 0 :(得分：0)
我已设法使用原始Catboost代码和libcatboost.so完成所有工作。
Java接口函数定义如下。请注意，为了模拟浮点值和字符串的2D数组（或指向指针的指针），我使用的是Pointer[]类型。
public interface CatboostModel extends Library {
public boolean LoadFullModelFromFile(Pointer calcer, String filename);
public boolean CalcModelPrediction(Pointer calcer, size_t docCount,
Pointer[] floatFeatures, size_t floatFeaturesSize,
Pointer[] catFeatures, size_t catFeaturesSize,
Pointer result, size_t resultSize);
}
之后，我填充floatFeatures和catFeatures这样的参数（这里有一些虚拟数据）。请注意，对于字符串，我使用的是JNA的StringArray。
float[] floatFeatures = {0.4f, 0.8f, 0.3f, 0.3f, 0.1f, 0.5f, 0.4f, 0.8f, 0.3f, 0.3f} ;
String[] catFeatures = {"1", "2", "3", "4"};
final Pointer pFloatFeatures = new Memory(floatFeatures.length * Native.getNativeSize(Float.TYPE));
final Pointer[] ppFloatFeatures = new Pointer[2];
for (int dloop=0; dloop<10; dloop++) {
pFloatFeatures.setFloat(dloop * Native.getNativeSize(Float.TYPE), floatFeatures[dloop]);
}
ppFloatFeatures[0] = pFloatFeatures;
ppFloatFeatures[1] = pFloatFeatures;
final Pointer[] ppCatFeatures = new Pointer[catFeatures.length];
final Pointer pCatFeatures = new StringArray(catFeatures);
ppCatFeatures[0] = pCatFeatures;
ppCatFeatures[1] = pCatFeatures;
最后，我将这些参数传递给Catboost：
if (!catboost.CalcModelPrediction(
modelHandle,
new size_t(2L),
ppFloatFeatures, new size_t((long)floatFeatures.length),
ppCatFeatures, new size_t((long)catFeatures.length),
pResult, new size_t(2L)
))
{
throw new RuntimeException("No prediction made: " + catboost.GetErrorString());
}
为了得到预测，我们可以做到：
double[] result = pResult.getDoubleArray(0, 2);