Manipulating ASCII Data Files

There are often occasions when you want to look at, or manipulate an ASCII data file:

1. To get a quick look at the data,  to see what it is like and evaluate whether you can use.
2. To fix problems.

Always keep a copy of the data file in case you make a mistake.

You will have to decide if the data is delimited (commas or tabs are most common, with tabs tricky to recognize), or fixed width.  The import wizards in Excel and Matlab do a good job of figuring this out.

Problems you might have to address:

• The data file uses a letter, such as "T" for a trace of snow.  When all the other lines have an actual number, you will have problems.  You might solve this by replacing the "T" with a tiny number (say "0.0001") which will work for most things you might want to do, but you have to think about what you are going to do with the data to insure that is a workable solution.
• There are columns with text, which will limit your options in Matlab (such as not being able to import into vectors).  You could change those to a number and ignore them, delete the column if you don't need it, or make two separate data files, one with the text and one with the numbers, and import them separately, or finally figure out alternatives in Matlab for the mixed file.
• There are extra lines at the beginning and end of the file.
• The column headers have more than two lines, and you have to get a single line.

You have several options if you want to edit the file:

1. Notepad or Wordpad
   • These behave almost identically (word wrap on long lines is probably the only significant difference).
   • Little risk of saving in an incorrect format.
   • Global search and replace can be slow
2. Word
   • Must insure you do not save in Word format.
   • Global search and replace is fast
3. Excel
   • Global search and replace is fast, and if you select just a column, you can operate just in that column and worry about making changes elsewhere you do not want to.
   • Always save in CSV format, with at extension.
   • If you are sure you do not need a column of data, you can remove it.

last revised 2/4/2013