Using Vim to semi-automatically ingest contents of files

The title is vague. The post has to do with the workflow I came up with and used for something today. Let me just walk you through a hypothetical scenario and how to solve it. The tutorial is *Nix specific.

Imagine you have a bunch of files under a directory, with some plain text content, and you want to ingest the contents of the files into a single file. That by itself is easy to do with a simple $ cat * > target_file. But how about if you want to actually have the names of the files along with the contents, so you know where the contents came from, and maybe you also want to edit some of the content out and perhaps also, you most likely want the files to be ingested in some sort of a temporal ordering. For the sake of this example let us assume you want the files sorted in the chronological order – you want the contents of the oldest file to appear up top.

Can’t think of a scenario? How about having a bunch of notes, each written once a week, with the dates being the names of the files – and you want to conflate the notes into a single, master-notes file for the whole year, and you want the contents in the order that they were created, with the notes created on the oldest date appearing first.

Here’s how you do it with the combination of a shell and Vim.

Step 1: $ ls -ltr > master_file.txt – this will put a list of the files in the chronological order into the master_file.txt file. Note that just the ‘t’ flag is for the reverse chronological order, and you need to revert that using ‘r’

Step 2: Open the master_file.txt in Vim

Step 3: Get rid of the stuff you do not want using visual blocks or whatever Vim magic you know in a few keystrokes

Step 4: Also get rid of the name ‘master_file.txt’. Now you have a raw list of all the files whose contents you want. and the files are listed in the chronological order

Step 5: Start recording a macro (say, qa)

Step 6: Place the cursor at the beginning of the first line. Press yy to copy the first file’s name

Step 7: Press o<Enter> to make some space

Step 8: Hit <Esc> followed by :r <Ctrl-r>” to paste the contents of the filename you just copied from the default register

Step 9: You might have to press <Del> once to get rid of the ^M character that gets pasted

Step 10: Hit <Enter> and the contents of the file will be in the buffer at the appropriate place

Step 11: Place the cursor at the beginning of the next file’s name, and stop recording the macro (q)

Step 12: Got 1000 files? 1000@a (or whatever register you recorded it in) is your friend!

And trust me – if you are a Vim user, it’s much faster and simpler than it seems here in the long description. If you are a newbie, trust me – it’s must faster, simpler and funner than it seems.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s