Control M or Ctrl-M or ^M character whatever you call them creates the problem when present in Unix or Linux text file. The presence of these characters may lead to unexpected results or interrupt the working of shell script, configuration files, etc.
Unix /Linux is the most widely used operating system for server and database hosting. Developers create code, text script on Windows, and move those to Unix environment where script execution fails with Command not found and Program exited with status 127 error.
What is ^M character in Unix/Linux?
Ctrl M or ^M is the carriage return character. Those come in the file because of different line termination characters used by Unix and Windows/DOS operating systems. Unix uses only line feed (LF) while windows use both carriage return (CR) and line feed (LF) as termination characters. This carriage return (CR) character becomes Ctrl M in Unix/Linux.
See the below image for details.
Want to know more about line termination character, just head on to Wikipedia.
How to find ^M character
There are multiple ways to find out the CTRL-M character in a text file. You can use text editors like Notepad, Notepad++, Textpad, VI, or even cat/grep Unix commands should also do the job.
Below we have covered the methods,
1. Using Notepad++ text editor
Open the file in Notepad++. Click on View -> Show Symbol -> Show All Characters or click this “¶” menu on the toolbar to displays all the characters including CR and LF. As mentioned above Unix only uses LF while Windows uses CRLF as a line termination chacter. While moving the Windows file to Unix, you need to remove this CR character.
2. cat -v command
cat command with -v option displays non-printing characters including ^M on the standard output as shown below.
$ cat -v texthost.prog echo "hi how are you"^M ls^M ^M
3. grep command
grep command allows you to search a string in a file. So run the grep command on the file as shown below to find out and display all the lines where the ^M character is present.
To type “^M” – click Ctrl+V and Ctrl+M i.e you can hold the CTRL key and press V and M sequentially. V should be first.
$ grep ^M texthost.prog echo "hi how are you" ls
4. vi text editor
-b option available in the VI Editor displays this special character. Simply run vi -b <file name> to display ^M on the screen as shown below.
$ vi -b texthost.prog echo "hi how are you"^M ls^M ^M
How to Avoid ^M character
You can use text editors like Notepad++ and TextPad to create files specific to the respective operating system. FTP software also helps to do explicit character conversion. Let’s discuss all these methods in detail and see how you can avoid Ctrl ^M character from a file.
1. Notepad++ or Textpad editor
Editors like Notepad++ or TextPad generally have a built-in option to save the file in Unix or Windows format. Depending on the target operating system you should save the file to avoid the character issue. For Notepad++, navigate to Edit->EOL Conversion and chose the respective file format. Here you need to choose UNIX/OSX format.
2. FTP file in ASCII mode
sFTP or FTP software like WinSCP, FileZilla provides options to transfer files using either Binary or ASCII mode. Always, transfer text files in ASCII mode from Windows to Unit or vice-version to avoid this ^M character issue for explicit character conversion.
The ASCII Mode does explicit character conversion. Windows format file is automatically converted to Unix format file and vice versa. Binary Mode moves the file as is with all properties.
How to Remove ^M character
The best is to avoid it using the method mentioned above, but if you still end up with it, then use the below method to remove the ^M character.
1. Explicit character conversion using dos2unix and unix2dos
Unix operating system like Solaris provides built-in command/utility dos2unix and unix2dos to convert file from Unix to DOS and vice versa.
dos2unix converts the file from DOS to UNIX format and unix2dos converts the file from UNIX to DOS format. This is the way you can use it.
dos2unix dos_format.txt unix_format.txt or unix2dos unix_format.txt dos_format.txt
2. vi editor
Use the VI editor in binary mode to search and replace the ^M character with null as shown in the below code snippet.
$ vi -b texthost.prog echo "hi how are you"^M ls^M ^M Press [ESC] Key and type below including : (do not copy type it) :%s/^M//g :wq $ cat -v texthost.prog echo "hi how are you" ls
3. sed command
You can also use the stream editor (sed) to find and replace ^M with space as shown in the below example.
$ sed -e "s/^M//g" texthost.prog > texthost.prog2 $ cat -v texthost.prog2 echo "hi how are you" ls mv texthost.prog2 texthost.prog
You can use the Perl search and replace to remove ^M and replace it with blank space. Check the below example.
$perl -pe 's/^M//g' texthost.prog > dummy $ cat -v dummy echo "hi how are you" ls mv dummy texthost.prog
If you do not want to use the mv command, run below
perl -i -pe 's/^M//g' texthost.prog
You can also create a backup of the original file using the -i.bak option.
$perl -i.bak -pe 's/^M//g' texthost.prog $ cat -v texthost.prog echo "hi how are you" ls $ cat -v texthost.prog.bak echo "hi how are you"^M ls^M ^M
5. tr command
Tr command with ‘-d’ option deletes the pattern instead of translating. Here, we deleting the ‘\r’ (non-printable) character.
tr -d '\r' < texthost.prog > texthost.prog2