Question

我正在尝试重现一个监视文件夹的控制台应用程序，并且该文件夹的任何新增文档都将被索引到ES。

工作正常如果我一次移动/添加3-4个文档并且能够编制索引。但是，如果我一次移动大约30个文档，它不会索引所有文档，而只索引一个文档。但是，如果我使用断点运行代码，那么甚至30个文档也会被编入索引。有人可以帮助我解决这个问题。

 static void OnCreated(object sender, FileSystemEventArgs e)
        {
            Console.WriteLine("File Created: Path: {0}, \n Name: {1}", e.FullPath, e.Name);
            Indexdoc(e.FullPath);
        }

如果我不在上面的代码中调用Indexdoc（e.FullPath）方法而是打印更改，它会显示所有添加的文件名。所以filesystemwatcher没有问题。我认为索引文档需要时间来产生响应并回到onCreated方法。

    public static void Indexdoc(string newFilePath)
            {
                List<Document> list = new List<Document>(); //list of Document class objects
                List<string> filesList = new List<string>(); //list of files in the path received on method call
                string path = string.Empty;

                client = ConfigSettings.connection();

                if (newFilePath == null) //for FULL Indexing
                {
                   //some code here
                }
                else //for new files indexing
                {
                    filesList.Add(newFilePath); //adds only one file everytime the method is called.
                    //the newFilePath will be of type C:/Documents/abc.txt
                }

                try
                {
                    foreach (string file in filesList)
                    {
                        Attachment attach = new Attachment
                        {
                            Name = Path.GetFileNameWithoutExtension(file),
                            Content = Convert.ToBase64String(File.ReadAllBytes(file)),
                            ContentType = Path.GetExtension(file)
                        };

                        var doc = new Document()
                        {
                            Title = Path.GetFileNameWithoutExtension(file),
                            FilePath = Path.GetFullPath(file), //added to get the path of the file
                            File = attach
                        };

                        list.Add(doc);
                    }
                }
                catch (Exception e)
                {
                    Console.WriteLine(e.Message.ToString());
                }

                var response = client.IndexMany(list, "trial");
            }

有人可以帮助我解决这个问题。

TIA

Answer 1

首先，创建一个您将用于放入信息的队列。这是在类范围内：

private BlockingCollection<string> CreatedQueue = new BlockingCollection<string>();

在启动时，您希望创建一个Task来监视该队列的更改：

var queueProcessor = Task.Factory.StartNew(() => ProcessQueue, TaskCreationOptions.LongRunning);

您的OnCreated事件处理程序如下所示：

static void OnCreated(object sender, FileSystemEventArgs e)
{
    Console.WriteLine("File Created: Path: {0}, \n Name: {1}", e.FullPath, e.Name);
    CreatedQueue.Add(e.FullPath);
}

ProcessQueue方法如下所示：

void ProcessQueue()
{
    foreach (var fileName in CreatedQueue.GetConsumingEnumerable())
    {
        Indexdoc(fileName);
    }
}

ProcessQueue方法将继续从队列中删除项目（或等待添加项目，以便可以删除它们），直到队列标记为已完成。也就是说，你说你已经完成了添加项目并且队列是空的。因此，当您关闭时，您必须调用CompleteAdding，然后在退出之前确保队列为空。为此，您的主程序执行此操作：

// Tell the queue that no more items will be added
CreatedQueue.CompleteAdding();

// Wait for the task to complete
queueProcessor.Wait();

Answer 2

我正在接受用户的输入以退出应用程序。我应该在下面添加这些代码吗？

// Tell the queue that no more items will be added
CreatedQueue.CompleteAdding();

// Wait for the task to complete
queueProcessor.Wait();

这是我修改过的代码。

    public class FileWatcher
    {
        public static ElasticClient client;
        public static FileSystemWatcher watcher;
        public static BlockingCollection<string> CreatedQueue = new BlockingCollection<string>();

        public static void Main(string[] args)
        {
            var queueProcessor = Task.Factory.StartNew(() => ProcessQueue(), TaskCreationOptions.LongRunning);
            watch();

        }

        public static void watch()
        {
            folderPath = ConfigurationManager.AppSettings["location"];


            try
            {
                watcher = new FileSystemWatcher();
                watcher.Path = folderPath;

                watcher.EnableRaisingEvents = true;
                watcher.IncludeSubdirectories = true;
                watcher.Filter = "*.*";


                watcher.Created += OnCreated;

                Console.WriteLine("*****************************************************************");
                // Wait for user to quit program.
                Console.WriteLine("Press \'q\' to quit the watcher application at any point of time.");
                Console.WriteLine();

                //Make an infinite loop till 'q' is pressed.
                while (Console.Read() != 'q') ;
            }
            catch (IOException e)
            {
                Console.WriteLine("A Exception Occurred :" + e);
            }

            catch (Exception oe)
            {
                Console.WriteLine("An Exception Occurred :" + oe);
            }
        }

        static void OnCreated(object sender, FileSystemEventArgs e)
        {
            Console.WriteLine("File Created: Path: {0}, \n Name: {1}", e.FullPath, e.Name);
            CreatedQueue.Add(e.FullPath);
        }

        public static void ProcessQueue()
        {
            foreach (var fileName in CreatedQueue.GetConsumingEnumerable())
            {
                Indexdoc(fileName);
            }
        }
    }

    public static void Indexdoc(string newFilePath)
    {}

文档索引为附件的FileSystemwatcher

2 个答案: