4

I need to write a shell script for converting characters in a file to lowercase and replace non alpha chars to _.

I have separate command for both the above. How to give a file as argument and make the above things done i.e. the output will have only lowercase chars and non alpha will be replace with _(underscore)?

  • 4
    Show what you have, someone will give you a solution. – user508889 Jun 24 '16 at 19:12
8

Using sed:

sed 's/[^[:alpha:]]/_/g; s/[[:upper:]]/\L&/g' file.txt

This is actually two sed commands separated by ;:

  • s/[^[:alpha:]]/_/g transforms each non-alphabetic character into a _

  • s/[[:upper:]]/\L&/g converts all uppercase characters into lowercases

Example:

$ cat file.txt 
fooBAr132323
SpamEgg

$ sed 's/[^[:alpha:]]/_/g; s/[[:upper:]]/\L&/g' file.txt
foobar______
spamegg
1

Python approach to this:

python -c 'import sys;print "\n".join(["".join([char.lower() if char.isalpha() else "_" for char in line]) for line in sys.stdin])' < input_file.txt

Basic idea here is that we split each line into individual characters and check if that char is alphabetical, then rebuild the line, and finally join all lines with newline to build a new file

Tested with /etc/passwd:

$ python -c 'import sys;print "\n".join(["".join([char.lower() if char.isalpha() else "_" for char in line]) for line in sys.stdin])' < /etc/passwd | head 

root_x_____root__root__bin_bash_
daemon_x_____daemon__usr_sbin__usr_sbin_nologin_
bin_x_____bin__bin__usr_sbin_nologin_
sys_x_____sys__dev__usr_sbin_nologin_
sync_x_________sync__bin__bin_sync_
games_x______games__usr_games__usr_sbin_nologin_
man_x______man__var_cache_man__usr_sbin_nologin_
lp_x_____lp__var_spool_lpd__usr_sbin_nologin_
mail_x_____mail__var_mail__usr_sbin_nologin_
news_x_____news__var_spool_news__usr_sbin_nologin_
0

The other answer solves your immediate problem, but a solution for the general case of "combining two commands" remains to be mentioned. Believe it or not, the Unix ecosystem was designed with this in mind, and your shell will definitely give you a way to do it.

If you want to feed a file through a "pipeline" of commands, you use the | (pipe) character in bash (presumably, the shell you are using).

Some other useful tools are worth mentioning here too: the "cat" utility, and the ">" redirection operator.

Let's assume your file is called input.txt, and you want to run it through your pipeline and call the result output.txt.

Let's build up our command piece by piece:

$ cat input.txt

Here's that "cat" I was talking about. For all intents and purposes, it just produces the content of your file. Now, let's add the first command in our pipeline: the "remove non-alphabet" part. I'll take a piece out of heemayl's sed command to do this:

$ cat input.txt | sed 's/[^[:alpha:]]/_/g'

Learning sed and regular expressions is a whole 'nother beast, so let's just be content that this command does what we want for now. Adding on the next piece:

$ cat input.txt | sed 's/[^[:alpha:]]/_/g' | tr 'A-Z' 'a-z'

I've taken a different approach than heemayl for this one, but it does the same job. Finally, let's put the output in the destination we want:

$ cat input.txt | sed 's/[^[:alpha:]]/_/g' | tr 'A-Z' 'a-z' > output.txt

There we go, we've done it. You can string as many commands together as you want with the | operator, then put the output in a file using >. This is a very common task in the command-line environment, so it's good to know what's going on and get used to using it. ;)

  • I need to write a shell script that takes file name as argument and converts all letters in the file to upper case and all non alpha characters to Underscore. – praveen reddy Jun 25 '16 at 12:22
  • I got- cat ws2.txt | sed "s/[^a-zA-Z]/ /g" | tr abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ | tr -s '[:space:]' '_' >output.txt – praveen reddy Jun 25 '16 at 12:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.