Reject URLs for -l/--list-fqdn and enforce FQDN input #18
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Problem\n\nThe shared CLI parser in client/lib/printout.py defines -l/--list-fqdn as a plain string option, so it accepts URLs even though the option contract says FQDN. Neither client/ccwget-local.py nor client/ccwget-remote.py rejects URL-shaped values before submitting the list-fqdn operation.\n\nConfirmed behavior:\n\n ./ccwget-remote.py -l https://www.thalesgroup.com\n\nsubmits the URL as the fqdn argument and reports no records. By contrast, -e is intended for exact URL occurrence searches and correctly accepts URLs.\n\nThis allows an invalid input to trigger a full dataset search with misleading results and inconsistent validation across FQDN-related options.\n\n## Proposed approach\n\n- Reuse the existing domain/FQDN validation convention from client/ccwget-local.py::valid_domain.\n- Apply validation to the shared -l/--list-fqdn parser option so local and remote clients reject schemes, paths, queries, fragments, and malformed FQDNs before any backend request.\n- Keep valid FQDN listing behavior unchanged.\n- Keep -e/--enumerate URL behavior unchanged.\n- Return a clear argparse error identifying that -l expects an FQDN, not a URL.\n- Add client contract tests for accepted FQDNs, URL rejection, malformed values, and unchanged -e behavior.\n- Update customer documentation if needed to clarify the distinction.\n\n## Scope\n\n- Modify the shared parser in client/lib/printout.py and the validation wiring in client/ccwget-local.py.\n- Ensure client/ccwget-remote.py receives the same validation through the shared parser.\n- Add deterministic tests with no live backend, Common Crawl, or ClickHouse dependency.\n- Document -l/--list-fqdn as FQDN-only and -e/--enumerate as URL-capable.\n- Preserve list-domain, domain-enumeration, PDNS validation, exact URL search, backend payload shape, and valid FQDN listing behavior.\n- Do not change backend /list-fqdn semantics or accept URLs for -l.\n\n## Acceptance criteria\n\n- -l www.thalesgroup.com is accepted and submits fqdn=www.thalesgroup.com.\n- -l https://www.thalesgroup.com is rejected client-side with a clear FQDN-not-URL error.\n- -l www.thalesgroup.com/path, -l www.thalesgroup.com?x=1, and fragment variants are rejected client-side.\n- Malformed or unsupported FQDN values are rejected before any HTTP request.\n- Local and remote clients expose identical validation behavior.\n- -e https://www.thalesgroup.com remains accepted and submits an exact URL search.\n- Automated tests cover success, URL rejection, malformed input, and regression behavior.\n- Existing client and server tests continue to pass.\n- Black and Pylint baselines do not regress.\n\n## Dependencies\n\nNone identified.
Fixed in commit
6d47a94. Shared -l/--list-fqdn parser now enforces FQDN validation; URLs, paths, queries, fragments, and malformed values reject client-side. Added local/remote regression tests and documentation.