我认为这是一个有趣的编程问题所以我发布了它,即使我认为我有一个足够好的解决方案,请参阅(*)。如果有人有一个优雅的解决方案,我很乐意看到它!
我正在使用一种方法来调用对服务器发出http请求的外部库。我需要让K字符串作为输入才有效,即外部资源的每次调用都是HTTP请求,我需要缓冲一些数据以提高效率。 (例如,让K为200,并且在文本中以1%的概率出现,因此我需要在找到200个输入参数之前处理20,000个令牌。)
有效的是:externalHTTP(commaDelimitedString) - >获取有关每个字符串的信息示例externalHTTP(“foo,bar”) - > [“信息片段1”,“信息片段2”]。 “信息片段1”的内容是“foo”。
我想用信息片段替换长文本(字符串)中的“foo”和“bar”,但只有在HTTP请求的缓冲区已满后才能替换。我仍然希望在等待这种情况发生时继续阅读原始字符串。
通过拆分对文本进行标记(因此我使用的是字符串数组)。
即。我不想因为等待K字符串缓冲而停止执行文本处理。
起初我以为我可以将单词作为单独的字符串对象存储,我稍后会更新但后来我意识到字符串是不可变的,因此它是按值调用的。
(*)我的第二个想法是存储单词的索引(foo和bar),然后在http请求完成时将片段插回到原始字符串数组中。像
class doStuff {
String text[];
LinkedList<Integer> idxList = new LinkedList<Integer>();
public doStuff(String[] t) {
text = t;
int i = 0;
for (String token : text) {
if (shouldReplaceToken(token)) {
replaceToken(i);
}
i++;
//do other work with the tokens
}
}
void replaceToken(int i) {
idxList.add(i);
if (++count > buffSize) {
count = 0;
String commaDelim = "";
ListIterator<Integer> it = idxList.getListIterator(0);
while (it.hasNext()) {
commaDelim += text[it.next()]+",";
}
String[] http_response = http_req(commaDelim);
for (String snippet : http_response) {
idx = idxList.poll(); //this is not elegant, dependent on FIFO order
text[Idx] = snippet;
}
}
}
}
进一步复杂化的是,我想处理几个较长的文本,所以我需要有一个String数组矩阵,每个文本一个。
我不喜欢已知的类参考
String[] text
或他们处理此代码中的索引......
希望看到一些建议:)
编辑:改变一下以便更清楚。我真的不能说我正在查找,不披露等,对不起。有些名称可能与java不同(只有很小的区别)。
答案 0 :(得分:1)
好的......这是尝试使用示例代码完全回答您的问题。
我从未玩过很多线程,所以我想我今晚会尝试学习一些东西。
此解决方案使用线程允许http请求异步发生。
使用Thread.sleep()模拟异步请求。
我的测试用例是原始的:主要课程只需要休息30秒,等待一切都完成。
就像我说的,我是线程编程的新手,所以我可能忽略了一些东西。
希望这能让你开始朝着正确的方向前进......
/**
* A class that asynchronously replaces text in an
* array of strings by using helper threads.
*/
public class TextReplacer {
private final int bufferSize;
List<String> wordList = new ArrayList<String>();
List<Integer> indexList = new ArrayList<Integer>();
int bufferPosition = 0;
int lastPosition = 0;
public TextReplacer(String[] a, int n) {
bufferSize = n;
if (a != null) {
wordList = Arrays.asList(a);
}
}
public void replaceText() {
int i = 0;
for (String thisWord : getWordListCopy()) {
if (shouldReplaceToken(thisWord)) {
indexList.add(i);
processTextReplacement();
}
i++;
}
}
private void processTextReplacement() {
if (isBufferReady()) {
int currentPos = lastPosition;
replaceStrings(getCsv(), currentPos);
}
}
/** Uses a thread to replace strings in wordList. */
private void replaceStrings(String csv, int pos) {
new ReplacerThread(wordList, indexList, csv, pos, bufferSize).start();
}
private String getCsv() {
StringBuilder csv = new StringBuilder();
for (int i = 0; i < bufferSize; i ++) {
int idx = indexList.get(lastPosition++);
csv.append(wordList.get(idx)).append(",");
}
return csv.toString();
}
private boolean isBufferReady() {
bufferPosition++;
return ( bufferPosition % bufferSize == 0 );
}
private List<String> getWordListCopy() {
List<String> listCopy = new ArrayList<String>();
listCopy.addAll(wordList);
return listCopy;
}
/**
* Simulates a 10% replacement rate by only
* returning true for input that ends with a 3.
*/
private boolean shouldReplaceToken(String s) {
return s.endsWith("3");
}
public List<String> getWordList() {
return wordList;
}
public String[] getWordArray() {
return wordList.toArray(new String[0]);
}
}
/**
* A thread that sleeps for up to 8 seconds, then
* replaces a bunch of words in the list that is
* passed to it in its constructor.
*/
public class ReplacerThread extends Thread {
List<String> originalWords;
List<Integer> indices;
String wordCsv;
String[] replacementWords;
int startPos;
int bufferSize;
int maxSleepMillis = 8000;
int sleepMillis = getSleepMillis();
int threadNum; // for debugging
String prefix = new String(); // for debugging
/** Create a new thread. */
public ReplacerThread(List<String> o, List<Integer> i,
String c, int p, int n) {
originalWords = o;
indices = i;
wordCsv = c;
startPos = p;
bufferSize = n;
threadNum = startPos / bufferSize;
int count = 0;
while (count++ < threadNum) {
prefix += " ";
}
}
@Override
public void run() {
replacementWords = httpReq(wordCsv);
for (int i = 0; i < bufferSize; i ++) {
int pos = startPos + i;
int idx = indices.get(pos);
originalWords.set(idx, replacementWords[i]);
}
print("Thread #" + threadNum + " COMPLETE");
}
/** Simulate an asynchronous http request by using Thread.sleep */
private String[] httpReq(String s) {
try {
printSleepMessage();
sleep(sleepMillis);
}
catch (InterruptedException ex) {}
String[] repText = s.split(",");
for (int i = 0; i < repText.length; i++) {
repText[i] = repText[i].replace("Line", "Funky Line");
}
return repText;
}
private void printSleepMessage() {
int ms = sleepMillis / 1000;
print("Thread #" + threadNum + " SLEEP(" + ms + ")");
}
private int getSleepMillis() {
Double ms = maxSleepMillis * Math.random();
return ms.intValue();
}
public void print(Object o) {
String s = (o == null ? "null" : o.toString());
System.out.println(prefix + s + "\n");
}
}
/** A class that tests my funky solution. */
public class Main {
static String inputFile = "test-input.txt";
static int bufferSize = 50;
public static void main(String[] args) {
String[] theInput = readInput();
TextReplacer testItem = new TextReplacer(theInput, bufferSize);
testItem.replaceText();
try {
// wait 40 seconds for everything to happen
Thread.sleep(40000);
}
catch (InterruptedException ex) { }
dumpOutput(testItem.getWordArray());
}
public static String[] readInput() {
File inFile = new File(inputFile);
List<String> lineList = new ArrayList<String>();
try {
BufferedReader buff = new BufferedReader(new FileReader(inFile));
String currentLine = buff.readLine();
while (currentLine != null) {
lineList.add(currentLine);
currentLine = buff.readLine();
}
}
catch (IOException ignoreMe) {}
print("Lines read: " + lineList.size());
return lineList.toArray(new String[0]);
}
public static void dumpOutput(String[] txt) {
long ms = System.currentTimeMillis();
String fileName = "output-" + ms + ".txt";
File outFile = new File(fileName);
try {
BufferedWriter buff = new BufferedWriter(new FileWriter(outFile));
for (String s : txt) {
buff.write(s);
buff.newLine();
}
}
catch (IOException ignoreMe) {}
print("Lines written: " + txt.length);
print("File: " + fileName);
}
public static void print(Object o) {
System.out.println(o == null ? "null" : o.toString());
}
}