Square brackets in filenames #

Since the BARs have GPSs in them, they encode the Latitude and Longitude in the folder and file names of their recordings. The problem is they do it using the following format: 20160916_Cast16-001 [-37.5397 141.3252]/20160916_154131_Cast16-001-Dsk [-37.5397 141.3252].wav. The square brackets in filenames are often used by shells (including Bash on Linux and PowerShell on Windows) to denote Range Wildcards. This means many common commands, scripts, and even our workbench fail to read the files unless the paths are treated as literals.

Status #

Minor Problem

Status with vendor #

Have contacted Frontier Labs, they seemed receptive to feedback on changing the character used to denote GPS coordinates. Unsure if they changed the default or not.

Effects of the problem on common tools #

Acoustics Workbench (Ecosounds) #

• Latest update: Slowly making our scripts more resilient to wildcard paths.

Bash/PowerShell #

Both of these shells will attempt read square brackets as wildcards.

Make sure any command that receives a path with square brackets in it is appropriately quoted.

Bad:

> sox --info 20160916_Cast16-001 [-37.5397 141.3252]/20160916_154131_Cast16-001-Dsk [-37.5397 141.3252].wav

Good:

> sox --info "20160916_Cast16-001 [-37.5397 141.3252]/20160916_154131_Cast16-001-Dsk [-37.5397 141.3252].wav"

For PowerShell many commandlets have a -LiteralPath argument that disables wildcards

Bad:

cd F:\Work\GitHub\emu\test\Fixtures\FL_BAR_LT\3.03_PreallocatedHeader\20181101_AAO [-27.3866 152.8761]
# PowerShell still attempts to parse the wildcards
cd  "F:\Work\GitHub\emu\test\Fixtures\FL_BAR_LT\3.03_PreallocatedHeader\20181101_AAO [-27.3866 152.8761]"
sox --info "F:\Work\GitHub\emu\test\Fixtures\FL_BAR_LT\3.03_PreallocatedHeader\20181101_AAO [-27.3866 152.8761]\20181101_060000_REC [-27.3866 152.8761].flac"

Good:

cd -LiteralPath "F:\Work\GitHub\emu\test\Fixtures\FL_BAR_LT\3.03_PreallocatedHeader\20181101_AAO [-27.3866 152.8761]"
sox --info "F:\Work\GitHub\emu\test\Fixtures\FL_BAR_LT\3.03_PreallocatedHeader\20181101_AAO [-27.3866 152.8761]\20181101_060000_REC [-27.3866 152.8761].flac"

Workarounds #

Command to replace square brackets in names (using rename packaged with “SUSE Linux Enterprise Server 12 SP2” v12.2):

find . -name "\*\\[\*" -type d -exec rename -v '[' \_ '{}' \\;
find . -name "\*\\]\*" -type d -exec rename -v ']' \_ '{}' \\;
find . -name "\*\\]\*" -type f -exec rename -v ']' \_ '{}' \\;
find . -name "\*\\[\*" -type f -exec rename -v '[' \_ '{}' \\;

Using rename packaged with Ubuntu 16.04:

find . -depth -name "*[*" -type d -exec rename -v 's/(\]|\[)/_/g' {} \;

For harvesting, it is only necessary to rename the directory. This script does this in a batch https://github.com/QutEcoacoustics/baw-private/blob/22e70967f86e7b7f83311e7a0263fcfc40a87c9d/Scripts/removeSquareBrackets.ps1