What this prompt does
This prompt produces awk and sed solutions for processing [input_file_type] files containing [data_description] that are [file_size], with the goal of [processing_goal]. It asks for an awk script extracting [target_fields] into [output_format] while handling quoted fields, multiline values, and inconsistent field counts; a sed pipeline doing [transformation_description] with extended regex and in-place editing; a combined pipeline with sort, uniq, paste, and join to achieve [pipeline_goal]; proper BEGIN/END blocks and associative arrays; performance guidance on when to switch to [alternative_tool] past [performance_threshold]; and before/after examples.
The variables make the solutions concrete to your data. [target_fields] and [output_format] define exactly what the awk script emits, [transformation_description] drives the sed work, and [file_size] plus [performance_threshold] determine when the answer recommends moving off coreutils to [alternative_tool]. The edge-case handling (quoted delimiters, multiline values) is what separates a real parser from a fragile one-liner.
When to use it
- Triaging a misbehaving Linux box where multi-gigabyte logs are the first stop
- Extracting
[target_fields]into a clean[output_format]from messy input - Normalizing inconsistent date formats with a sed pipeline
- Building a coreutils pipeline that aggregates and reports without a heavier tool
- Deciding when to stay in awk/sed versus switch to
[alternative_tool] - Extracting slow endpoints and anomalies before a dashboard would surface them
Example output
You get an awk script with BEGIN/END blocks, field-separator handling, and associative arrays that extracts [target_fields] into [output_format], a sed pipeline performing [transformation_description] with backup, a combined awk/sed/coreutils pipeline achieving [pipeline_goal], notes on awk versus sed versus grep and the [alternative_tool] threshold, and before/after sample input and output for each solution.
Pro tips
- State
[target_fields]precisely; awk extraction is only as good as knowing which columns and separators your[input_file_type]really uses - Use the in-place sed editing with backup, not without; an irreversible
[transformation_description]on production data is a bad place to skip the backup - Lean on associative arrays in the END block for aggregation rather than piping to sort/uniq when one awk pass can do it
- Respect the
[performance_threshold]guidance — past[file_size]thresholds, switching to[alternative_tool]like Miller or DuckDB beats fighting awk - Handle the quoted-delimiter and multiline edge cases explicitly; naive field splitting on CSV with embedded commas silently corrupts output
- Parallelize with xargs or GNU parallel only when the work is independent; aggregation across the whole file does not split cleanly
- Ask for the before/after examples the prompt offers; seeing sample input mapped to expected
[output_format]catches a wrong field separator far faster than running the script against real data - Set FS, OFS, and RS explicitly in the awk BEGIN block rather than relying on defaults; mismatched separators are the most common reason a pipeline silently produces empty or shifted fields on
[input_file_type]data
