JIYIK CN >

This article demonstrates how to identify and correct unexpected token syntax errors due to invisible characters in Bash scripts.

Syntax Error Near Unexpected Token in Bash

Sometimes, a syntactically-seeming Bash script just doesn’t work. You’ll get an obscure syntax error, which doesn’t help.

Suppose you have some code like:

#!/bin/bash
while :
do
echo "This will repeat every second until you press CTRL + C"
sleep 1
done

Syntactically, the above code is correct. However, some invisible characters may make the code syntactically unreadable in Linux.

How can we confirm that invisible characters are causing the problem? How can we fix this?

As mentioned in the previous sections, the solution is simple.

Identify the problem

Assume that filename.sh is the file you saved for your Bash script.

Open your terminal and navigate to the file location. Then, run the following command in the terminal.

cat -v filename.sh

Invisible characters that might represent a carriage return or non-breaking space should show up as ^M、M-BM-or M-. If any other weird characters make their way into your file, you should be able to see them as well.

In our case we had several options that could solve our problem.

Remove Windows line breaks using console commands

If you have a problematic file named filename.sh, we can use the following command to save the corrected contents into a file named correctedFile.sh.

tr -d '\r' < filename.sh > correctedFile.sh

Automatically convert non-UNIX files to UNIX-friendly format

To do this, you need a utility called dos2unix. If you are using apt as your package manager, you can install dos2unix with the following command:

sudo apt install dos2unix

Alternatively, if you are using an arch based distribution, you can install dos2unix from the AUR.

To convert a file to UNIX friendly format, open a terminal and navigate to the folder containing the desired file. We will consider a file called filename.sh.

Enter the following command in your terminal.

dos2unix filename.sh

The above command will convert your file into UNIX friendly format. That's it, the problem is solved.

Hopefully, your Bash script will now run fine. If it still doesn't run, you most likely have an actual syntax problem.