Question

I have a little issue here. I want to read from a text file using python and create queue and then send these lines from the text file into Amazon web services SQS(Simple Queue service). First of all, I`ve actually managed to do this using boto, but the problem is that the lines dont come in order, just randomly, like line 4,line 1, line 5 etc etc.. Here is my code:

import boto.sqs

conn = boto.sqs.connect_to_region("us-east-2",
                                  aws_access_key_id='AKIAIJIQZG5TR3NMW3LQ',
                                  aws_secret_access_key='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')

q = conn.create_queue('test')
with open('read.txt', 'r') as read_file:
    from boto.sqs.message import RawMessage
    for line in read_file:
        m = RawMessage()
        m.set_body(line)
        q.write(m)

So, what to do? Well, we need to create an FIFO queue(which I also managed to do using boto3 in python), but now the problem is that I have problems reading the text file.. Here is the code i used to create a FIFO queue in SQS:

import boto3

AWS_ACCESS_KEY = 'AKIAIJIQZG5TR3NMW3LQ'
AWS_SECRET_ACCESS_KEY = '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'
sqs_client = boto3.resource(
    'sqs',
    aws_access_key_id=AWS_ACCESS_KEY,
    aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
    region_name='us-east-2'
)

queue_name = ('demo_queue.fifo')

response = sqs_client.create_queue(
    QueueName=queue_name,
    Attributes={
        'FifoQueue': 'true',
        'ContentBasedDeduplication': 'true'
    }
)
with open('read.txt', 'r') as read_file:
    from boto.sqs.message import RawMessage
    for line in read_file:
        m = RawMessage()
        m.set_body(line)
        queue_name.write(m)

Somebody know how to solve this? thanks.

Answer 1

尝试更换 queue_name.write(m) 同 response.write(m) 在你的第一段代码中。您应该使用get_queue_by_name返回的实际队列此外，仅在boto3中指定MessageBody和MessageGroupID时，请确保为队列启用了基于内容的重复数据删除，或者指定了MessageDeduplicationId字符串，否则将失败

read from text file and send in to aws sqs Fifo queue

1 个答案: