Automating Recon

The port scanner from build-a-port-scanner works. Hand it a host and it reports which ports answer. But it is a toy in three specific ways. The target is a string typed near the top of the file. The results scroll past in the terminal and then vanish. And scanning a second host means editing the source and running it again. A real recon tool fixes all three: it takes the target as an argument, reads a whole list of hosts from a file, and leaves behind a results file the next tool can open. This lesson is the gap between a script and a tool, and it is mostly plumbing that Python's standard library already wrote.

A script is not a tool yet

When a program is launched from a shell, the shell hands it a list of words: the program's own name, then everything typed after it. Python exposes that list as sys.argv. Reading it by hand is fine for one argument and turns into a swamp by the third, because the script then has to check how many words arrived, remember which position means what, convert the text into numbers, and print its own error message when someone gets it wrong.

argparse is the standard library module that does all of that. It is given a description of the arguments a program accepts, and in return it reads sys.argv, validates it, converts the values, and generates a help screen for free. Nothing gets installed; import argparse is enough.

Arguments come in two shapes, and the difference is worth naming. A positional argument is identified by where it sits in the command and is required by default: recon targets.txt passes targets.txt as the first positional value. An option starts with a dash, is identified by name rather than position, and is usually optional: -p 1-1024 or --ports 1-1024. A flag is an option that takes no value and is just present or absent, like --verbose.

Here is the scanner given a real interface. The scan() function is the port scanner from the previous lesson: given a host and a port range, it returns the list of open ports.

import argparse

def build_parser():
    parser = argparse.ArgumentParser(
        prog="recon",
        description="Scan hosts for open TCP ports.",
    )
    parser.add_argument("targets", help="file with one host per line")
    parser.add_argument("-p", "--ports", default="1-1024",
                        help="port range to scan (default: 1-1024)")
    parser.add_argument("-o", "--output",
                        help="write results to this JSON file")
    parser.add_argument("--delay", type=float, default=0.0,
                        help="seconds to wait between hosts (default: 0)")
    return parser

args = build_parser().parse_args()

Each add_argument call teaches the parser one thing. default sets the value used when the option is absent. type=float tells the parser to convert the text "0.5" into the number 0.5 before the script ever sees it, so no manual conversion is needed later. help is the one-line description that shows up on the help screen. The call to parse_args() reads sys.argv, applies every rule, and returns an object whose attributes hold the parsed values: args.targets, args.ports, args.delay. If a required argument is missing or an unknown option appears, parse_args() prints a usage message and exits before the rest of the script runs.

The help screen is generated, not written. Passing -h or --help triggers it.

$python recon.py --help

Every line came from the parser definition. The usage: line shows optional arguments in square brackets and the required targets bare, which is exactly the convention every other command-line tool uses. One version wrinkle to expect: on Python 3.9 and earlier the header reads optional arguments: instead of options:. Same meaning, older label.

Targets belong in a file, not the source

A recon run is rarely one host. The target list belongs in a plain text file, one host per line, so the same tool serves a single machine or a thousand without anyone touching the code. Reading that file is a few lines.

def read_targets(path):
    with open(path) as f:
        return [
            line.strip()
            for line in f
            if line.strip() and not line.startswith("#")
        ]

open(path) returns a file object. Iterating over it yields one line at a time, each still carrying its trailing newline character, so strip() removes the newline and any stray spaces. The condition keeps a line only if something is left after stripping and it does not start with #, which drops blank lines and lets the file carry comments. The with block closes the file automatically when the list is built.

Now the scan touches the network, and the network fails in ordinary ways: a host is powered off, a name does not resolve, a connection is refused. Left unguarded, the first of those failures raises an exception that ends the entire program, and every host listed after it goes unscanned. Wrapping the per-host work in a try/except turns a fatal error into a recorded one, and the loop keeps going.

import sys

results = []
for host in read_targets(args.targets):
    try:
        open_ports = scan(host, args.ports)
        results.append({"host": host, "open_ports": open_ports})
    except OSError as err:
        print(f"{host}: {err}", file=sys.stderr)
        results.append({"host": host, "error": str(err)})

The exception to catch is OSError. Every socket failure raises it or one of its subclasses: socket.gaierror when a hostname does not resolve, ConnectionRefusedError when a port slams the door, TimeoutError when nothing answers in time. Catching the parent OSError catches all of them in one line, so a bad host is logged and the run moves on. The file=sys.stderr argument sends the error text to the standard error stream instead of standard output, which keeps the error noise out of any results that get redirected to a file.

Output a machine can read next

Printed text is for a human reading right now. A recon tool's output almost always feeds another step: a service scanner, a report, a second pass. That means the output should be structured, not a wall of prose to be re-parsed later. Two formats cover nearly everything, and both live in the standard library.

JSON keeps the shape. json.dump writes a Python object, the list of dictionaries built above, to a file as JSON text. indent=2 makes it readable instead of one long line.

import json

if args.output:
    with open(args.output, "w") as f:
        json.dump(results, f, indent=2)

Any tool that speaks JSON, including a later run of this same script, reads it straight back with json.load and gets the identical structure: the list, the dictionaries, the port numbers as real integers. Nothing is re-parsed from text.

That [Errno -2] Name or service not known is the real text of a DNS failure on Linux; macOS prints a different wording for the same problem. It landed in the file instead of the terminal because the earlier except block stored it, which is why a good scan and a dead host sit side by side in the same record.

CSV is the other workhorse. When the result flattens into a table, a comma separated file opens in any spreadsheet and pipes into other command-line tools. The csv module handles the quoting and escaping that make hand-built CSV a recurring bug.

import csv

with open("results.csv", "w", newline="") as f:
    writer = csv.writer(f)
    writer.writerow(["host", "port"])
    for row in results:
        for port in row.get("open_ports", []):
            writer.writerow([row["host"], port])

One detail that bites everyone once: the newline="" on the open call is required. Without it, csv.writer and the file object both add line endings on some platforms, and the output gets a blank row between every real one. The row.get("open_ports", []) returns an empty list for any host that errored, so those hosts simply contribute no rows.

Rate limiting, and not being the loud one

The scanner can open connections as fast as a host will answer. Against a real target that is a problem for two separate reasons. A burst of connections to many ports in a second looks exactly like an attack to any intrusion detection system, so it trips alarms and gets the source address blocked. And a flood of half-open connections can exhaust the resources of a small or fragile host, which crosses the line from testing a service into knocking it over.

A deliberate pause fixes both. time.sleep(seconds) stops the interpreter for that many seconds and does nothing else. Wiring it to the --delay option that argparse already parsed lets the pace be tuned from the command line without editing code.

import time

for host in read_targets(args.targets):
    scan_and_record(host)
    time.sleep(args.delay)

A tenth of a second between hosts is invisible to a human waiting and enormous to a firewall counting connections per second. The right value depends on the target, which is the honest answer: slow and quiet by default, faster only when the target owner has said the load is fine.

Rate limiting is one piece of a larger habit, which is being a scanner that a defender can live with. Scan only hosts that are owned or covered by written authorization, and confirm the target file matches that scope before every run. Where the protocol allows identifying the source, such as a User-Agent string on web recon, use one that names the tester, so a defender who notices the traffic can send an email instead of paging an incident response team at 2 a.m. And keep a record of what was scanned and when, because accountable activity is the whole difference between authorized testing and an intrusion.

Everything in this module now points at the same place: a working tool that takes a target list, scans it, survives dead hosts, and writes structured output a defender or a next tool can read. The last step is the one this lesson kept gating with warnings. Point it at real targets, the ones under written scope, and run recon the way an engagement actually runs it: quietly, on a schedule, with the results in a file instead of in somebody's head.