Category Archives: UNIX

UNIX tip of the day —
duplicate and replace lines with awk

Today I got some data I wanted to add to my machine learning training datasets for named entity recognition. My system is designed to be used with output from automatic speech recognition (ASR). It is frequently difficult to be certain whether ASR output will contain hyphens or not, e.g. (email, vs e-mail) so frequently I […]

Posted in linguistics, UNIX | Comments Off on UNIX tip of the day —
duplicate and replace lines with awk

UNIX tip – xargs with multiple commands

Xargs is an extremely powerful complement to the awesome find command. One downside is that you usually need to have a single pipeline. By default you can’t put together a bunch of commands which are not piped. However, it is possible to call a shell with xargs. In this way, you can execute multiple commands […]

Posted in UNIX | Comments Off on UNIX tip – xargs with multiple commands

Using awk to sum rows of numbers

I have a script which takes a tab-delmited file for regression tests, and converts it xml. I want to do a sanity check, to make sure that the number of utterances in my xml files matches the number in the tab-delimited.txt file. I can do this in 2 lines in UNIX robert_felty$ wc -l samples2.txt […]

Posted in bash, linux, UNIX | Comments Off on Using awk to sum rows of numbers