我当时正在使用scrapy-redis抓取数据并使用redis进行dupefilter请求，在抓取大约3000k请求之后，无论我重新启动Spider（无论是否清理键），蜘蛛都无法获取或将请求放入dupefilter密钥中或redis，它只会停止抓取。我想知道redis有一定的局限性吗？
我尝试了很多次，总是请求数约为3000k，它必须具有一定的关联性，请帮助我！
redis服务器显示：
[8096] 10 Jan 16:29:52 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:29:52 - 5 clients connected (0 slaves), 28787472 bytes in use
[8096] 10 Jan 16:29:57 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:29:57 - 5 clients connected (0 slaves), 28754120 bytes in use
[8096] 10 Jan 16:30:02 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:30:02 - 5 clients connected (0 slaves), 28585192 bytes in use
[8096] 10 Jan 16:30:07 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:30:07 - 5 clients connected (0 slaves), 27473288 bytes in use
[8096] 10 Jan 16:30:12 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:30:12 - 5 clients connected (0 slaves), 26759176 bytes in use
[8096] 10 Jan 16:30:17 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:30:17 - 5 clients connected (0 slaves), 28058760 bytes in use
[8096] 10 Jan 16:30:22 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:30:22 - 5 clients connected (0 slaves), 26286480 bytes in use
[8096] 10 Jan 16:30:24 * 10000 changes in 60 seconds. Saving...
[8096] 10 Jan 16:30:24 * Foregroud saving started by pid 8096
[8096] 10 Jan 16:30:24 * DB saved on disk
[8096] 10 Jan 16:30:24 * Background saving terminated with success
[8096] 10 Jan 16:30:27 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:30:27 - 5 clients connected (0 slaves), 25357000 bytes in use
[8096] 10 Jan 16:30:32 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:30:32 - 5 clients connected (0 slaves), 25802800 bytes in use
[8096] 10 Jan 16:30:37 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:30:37 - 5 clients connected (0 slaves), 23736768 bytes in use
[8096] 10 Jan 16:30:42 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:30:42 - 5 clients connected (0 slaves), 23954720 bytes in use
[8096] 10 Jan 16:30:47 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:30:47 - 5 clients connected (0 slaves), 22061280 bytes in use
[8096] 10 Jan 16:30:52 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:30:52 - 5 clients connected (0 slaves), 21993872 bytes in use
[8096] 10 Jan 16:30:57 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:30:57 - 5 clients connected (0 slaves), 22569824 bytes in use
[8096] 10 Jan 16:31:03 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:31:03 - 5 clients connected (0 slaves), 21005816 bytes in use
[8096] 10 Jan 16:31:08 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:31:08 - 5 clients connected (0 slaves), 20970824 bytes in use
[8096] 10 Jan 16:31:13 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:31:13 - 5 clients connected (0 slaves), 19985472 bytes in use
[8096] 10 Jan 16:31:18 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:31:18 - 5 clients connected (0 slaves), 19297080 bytes in use
[8096] 10 Jan 16:31:23 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:31:23 - 5 clients connected (0 slaves), 19221384 bytes in use
[8096] 10 Jan 16:31:28 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:31:28 - 5 clients connected (0 slaves), 16717416 bytes in use
[8096] 10 Jan 16:31:30 * 10000 changes in 60 seconds. Saving...
[8096] 10 Jan 16:31:30 * Foregroud saving started by pid 8096
[8096] 10 Jan 16:31:31 * DB saved on disk
[8096] 10 Jan 16:31:31 * Background saving terminated with success
[8096] 10 Jan 16:31:33 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:31:33 - 5 clients connected (0 slaves), 16596904 bytes in use
[8096] 10 Jan 16:31:38 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:31:38 - 5 clients connected (0 slaves), 17379168 bytes in use
[8096] 10 Jan 16:31:43 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:31:43 - 5 clients connected (0 slaves), 17069712 bytes in use
[8096] 10 Jan 16:31:48 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:31:48 - 5 clients connected (0 slaves), 15774568 bytes in use
[8096] 10 Jan 16:31:53 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:31:53 - 5 clients connected (0 slaves), 14270800 bytes in use
[8096] 10 Jan 16:31:58 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:31:58 - 5 clients connected (0 slaves), 14275816 bytes in use
[8096] 10 Jan 16:32:03 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:32:03 - 5 clients connected (0 slaves), 10735304 bytes in use
[8096] 10 Jan 16:32:08 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:32:08 - 5 clients connected (0 slaves), 10739464 bytes in use
[8096] 10 Jan 16:32:13 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:32:13 - 5 clients connected (0 slaves), 10825088 bytes in use
[8096] 10 Jan 16:32:18 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:32:18 - 5 clients connected (0 slaves), 10661624 bytes in use
[8096] 10 Jan 16:32:23 - Accepted 127.0.0.1:53991
[8096] 10 Jan 16:32:23 - Accepted 127.0.0.1:53992
[8096] 10 Jan 16:32:23 - DB 0: 1 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:32:23 - 7 clients connected (0 slaves), 7264584 bytes in use
[8096] 10 Jan 16:32:28 - DB 0: 1 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:32:28 - 7 clients connected (0 slaves), 7298664 bytes in use
[8096] 10 Jan 16:32:34 - DB 0: 1 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:32:34 - 7 clients connected (0 slaves), 7318112 bytes in use
[8096] 10 Jan 16:32:39 - DB 0: 1 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:32:39 - 7 clients connected (0 slaves), 7332752 bytes in use
[8096] 10 Jan 16:32:44 - DB 0: 1 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:32:44 - 7 clients connected (0 slaves), 7342720 bytes in use
[8096] 10 Jan 16:32:44 - Client closed connection
[8096] 10 Jan 16:32:44 - Client closed connection
[8096] 10 Jan 16:32:44 - Client closed connection
[8096] 10 Jan 16:32:44 - Client closed connection
[8096] 10 Jan 16:32:44 - Client closed connection
[8096] 10 Jan 16:32:44 - Client closed connection
[8096] 10 Jan 16:32:49 - DB 0: 1 keys (0 volatile) in 4 slots HT.
[8096] 10 Jan 16:32:49 - 1 clients connected (0 slaves), 4964808 bytes in use
[8096] 10 Jan 16:32:54 - DB 0: 1 keys (0 volatile) in 4 slots HT.
当错误出现时，它突然又有两个客户端连接，我不知道为什么会发生 和他们我关闭了蜘蛛和客户端关闭的连接