Question

我正在尝试构建一个子域强力执行器以供我的客户使用 - 我从事安全/笔测试工作。目前，我能够让Resolv在10秒内查找大约70个主机，给予或接受并想知道是否有办法让它做更多。我已经看到了替代脚本，主要是基于Python，可以实现比这更快的速度。我不知道如何增加Resolv并行发出的请求数，或者我是否应该将列表拆分。请注意我已将Google的DNS服务器放在示例代码中，但将使用内部DNS服务器进行实时使用。

我调试此问题的粗略代码是：

require 'resolv'

def subdomains
  puts "Subdomain enumeration beginning at #{Time.now.strftime("%H:%M:%S")}"
  subs = []
  domains = File.open("domains.txt", "r") #list of domain names line by line.
  Resolv.new(:nameserver => ['8.8.8.8', '8.8.4.4'])
    File.open("tiny.txt", "r").each_line do |subdomain|
      subdomain.chomp!
    domains.each do |d|
      puts "Checking #{subdomain}.#{d}"
      ip = Resolv.new.getaddress "#{subdomain}.#{d}" rescue ""
        if ip != nil
          subs << subdomain+"."+d << ip
      end
    end
  end
  test = subs.each_slice(4).to_a
    test.each do |z|
      if !z[1].nil? and !z[3].nil?
    puts z[0] + "\t" + z[1] + "\t\t" + z[2] + "\t" + z[3]
  end
end
  puts "Finished at #{Time.now.strftime("%H:%M:%S")}"
end

subdomains

domains.txt是我的客户端域名列表，例如google.com，bbc.co.uk，apple.com和'tiny.txt'是潜在的子域名列表，例如ftp，www，dev ，文件，上传。然后Resolv将查找files.bbc.co.uk，并告诉我它是否存在。

Answer 1

有一件事是你正在使用Google名称服务器创建一个新的Resolv实例，但从不使用它;您创建一个全新的Resolv实例来执行getaddress调用，因此该实例可能使用一些默认的名称服务器而不是Google的名称服务器。您可以将代码更改为以下内容：

resolv = Resolv.new(:nameserver => ['8.8.8.8', '8.8.4.4'])
# ...
ip = resolv.getaddress "#{subdomain}.#{d}" rescue ""

此外，我建议使用File.readlines方法来简化代码：

domains = File.readlines("domains.txt").map(&:chomp)
subdomains = File.readlines("tiny.txt").map(&:chomp)

此外，你正在拯救坏的ip并将其设置为空字符串，但是在下一行中你测试的不是nil，所以所有结果都应该通过，我不会认为＆＃ 39; s你想要的。

我已经重构了您的代码，但未对其进行测试。这是我想出来的，可能更清楚：

def subdomains
  puts "Subdomain enumeration beginning at #{Time.now.strftime("%H:%M:%S")}"
  domains = File.readlines("domains.txt").map(&:chomp)
  subdomains = File.readlines("tiny.txt").map(&:chomp)

  resolv = Resolv.new(:nameserver => ['8.8.8.8', '8.8.4.4'])

  valid_subdomains = subdomains.each_with_object([]) do |subdomain, valid_subdomains|
    domains.each do |domain|
      combined_name = "#{subdomain}.#{domain}"
      puts "Checking #{combined_name}"
      ip = resolv.getaddress(combined_name) rescue nil
      valid_subdomains << "#{combined_name}#{ip}" if ip
    end
  end

  valid_subdomains.each_slice(4).each do |z|
    if z[1] && z[3]
      puts "#{z[0]}\t#{z[1]}\t\t#{z[2]}\t#{z[3]}"
    end
  end

  puts "Finished at #{Time.now.strftime("%H:%M:%S")}"
end

此外，您可能想要查看dnsruby gem（https://github.com/alexdalitz/dnsruby）。它可能会比Resolv做得更好。

Answer 2

[注意：我已经重写了代码，以便以块的形式获取IP地址。请参阅https://gist.github.com/keithrbennett/3cf0be2a1100a46314f662aea9b368ed。您可以修改RESOLVE_CHUNK_SIZE常量以平衡性能和资源负载。]

我使用dnsruby gem重写了这段代码（主要由Alex Dalitz在英国撰写，并由我和其他人贡献）。此版本使用异步消息处理，以便几乎同时处理所有请求。我已在https://gist.github.com/keithrbennett/3cf0be2a1100a46314f662aea9b368ed发布了一个要点，但也会在此处发布代码。

请注意，由于您不熟悉Ruby，因此代码中有许多内容可能对您有所帮助，例如方法组织，Enumerable方法的使用（例如惊人的“分区”方法），Struct类，抢救一个特定的异常类，％w和Benchmark。

注意：看起来像堆栈溢出执行最大消息大小，因此该代码被截断。转到完整代码的链接上的GIST。

#!/usr/bin/env ruby

# Takes a list of subdomain prefixes (e.g.  %w(ftp  xyz)) and a list of domains (e.g. %w(nytimes.com  afp.com)),
# creates the subdomains combining them, fetches their IP addresses (or nil if not found).

require 'dnsruby'
require 'awesome_print'

RESOLVER = Dnsruby::Resolver.new(:nameserver => %w(8.8.8.8  8.8.4.4))

# Experiment with this to get fast throughput but not overload the dnsruby async mechanism:
RESOLVE_CHUNK_SIZE = 50


IpEntry = Struct.new(:name, :ip) do
  def to_s
    "#{name}: #{ip ? ip : '(nil)'}"
  end
end


def assemble_subdomains(subdomain_prefixes, domains)
  domains.each_with_object([]) do |domain, subdomains|
    subdomain_prefixes.each do |prefix|
      subdomains << "#{prefix}.#{domain}"
    end
  end
end


def create_query_message(name)
  Dnsruby::Message.new(name, 'A')
end


def parse_response_for_address(response)
  begin
    a_answer = response.answer.detect { |a| a.type == 'A' }
    a_answer ? a_answer.rdata.to_s : nil
  rescue Dnsruby::NXDomain
    return nil
  end
end


def get_ip_entries(names)

  queue = Queue.new

  names.each do |name|
    query_message = create_query_message(name)
    RESOLVER.send_async(query_message, queue, name)
  end


  # Note: although map is used here, the record in the output array will not necessarily correspond
  # to the record in the input array, since the order of the messages returned is not guaranteed.
  # This is indicated by the lack of block variable specified (normally w/map you would use the element).
  # That should not matter to us though.
  names.map do
    _id, result, error = queue.pop
    name = _id
    case error
      when Dnsruby::NXDomain
        IpEntry.new(name, nil)
      when NilClass
       ip = parse_response_for_address(result)
       IpEntry.new(name, ip)
      else
       raise error
      end
  end
end


def main
  # domains = File.readlines("domains.txt").map(&:chomp)
  domains = %w(nytimes.com  afp.com  cnn.com  bbc.com)

  # subdomain_prefixes = File.readlines("subdomain_prefixes.txt").map(&:chomp)
  subdomain_prefixes = %w(www  xyz)

  subdomains = assemble_subdomains(subdomain_prefixes, domains)

  start_time = Time.now
  ip_entries = subdomains.each_slice(RESOLVE_CHUNK_SIZE).each_with_object([]) do |ip_entries_chunk, results|
    results.concat get_ip_entries(ip_entries_chunk)
  end
  duration = Time.now - start_time

  found, not_found = ip_entries.partition { |entry| entry.ip }

  puts "\nFound:\n\n";  puts found.map(&:to_s);  puts "\n\n"
  puts "Not Found:\n\n"; puts not_found.map(&:to_s); puts "\n\n"

  stats = {
      duration:        duration,
      domain_count:    ip_entries.size,
      found_count:     found.size,
      not_found_count: not_found.size,
  }

  ap stats
end


main

提高Ruby Resolv Speed

2 个答案: