Friday, December 7, 2012

Keep unique lines in file using only first field of the line

To keep unique lines in the file but only using the first field of the line. Note that the first line in the file will be kept the rest will be discarded.
 
awk '!x[$1]++' filename > filename2

e.g.

Myh7    -3.41856493500000000000 0.00000000005092670000 
Mest    -2.74194127200000000000 0.00003431680000000000

Mest    -2.67886271200000000000 0.00005941270000000000 
Mest    -2.45088556100000000000 0.00000001772790000000
Mest    -2.43988796300000000000 0.00000001257030000000
Mest    -2.41913470400000000000 0.00000000293480000000
Mest    -2.41640532100000000000 0.00000001299950000000
Mest    -2.40872435300000000000 0.00000001836740000000
Mest    -2.37917197000000000000 0.00000002830130000000
Mest    -2.37905761900000000000 0.00000002079510000000

Mest    -2.74194127200000000000 0.00003431680000000000
Pde4dip -2.70488951700000000000 0.00000185955000000000  
...

will give:

Myh7    -3.41856493500000000000 0.00000000005092670000
Mest    -2.74194127200000000000 0.00003431680000000000
Pde4dip -2.70488951700000000000 0.00000185955000000000
...

No comments:

Post a Comment