25 Sep

UTF-8 with BOM

When localizing DragonScales 3 we experienced a baffling issue with an internal tool whose purpose is simply to replace text in a group of files. Those UTF-8 encoded files contain messages loaded by the game from the very beginning. However, after running the tool, the game started crashing when reading such files. By using an old buddy, fc /B, we found out that our tool was “injecting” a few extra bytes at the start of the file: EF BB BF. In short, the tool was altering the encoding of files from UTF-8 to UTF-8 with BOM. That was the cause for the crashing, as our game expects the files to be UTF-8 encoded without BOM.

What’s this BOM, anyway? Simply put, it’s just a sequence of bytes (EF BB BF) used to signal readers about the file being UTF-8 encoded. It seems such mark might be useful in some specific contexts, with some specific programs. Not our case, so we had to remove the BOM with a little batch script like this:

for /r ".\DE\scenes" %%i in (*.*) do (
  copy %%i .\tmp.txt /Y
  sed -i '1s/^\xEF\xBB\xBF//' .\tmp.txt
  attrib -R .\tmp.txt
  move /Y .\tmp.txt %%i
)

In this snippet we remove the BOM via sed. Files are those under a fictitious directory, .\DE\scenes. Those copies and attribs help to circumvent some problems with permissions of files created by our sed version on Windows.

31 Aug

Extracting text from PSD files

Typically, we need to translate hundreds of strings when localizing our games. Most strings are text messages which the game loads from some database or simple text file. However, we often have to handle localization of several PNG images, such as the one below.

AWESOME! message in DragonScales

Such PNG images are exported from PSD files which must obviously contain at least one Text Layer. To speed up the localization process we have a little Photoshop script which opens the PSD files and extracts all the text we have to translate. PSD files are grouped in directories corresponding to the tileset they belong to. For instance, this would be a typical directory structure for the DragonScales games:

images/
    board/
        awesome.psd, great.psd, ...
    cards
        youwin.psd, sorry.psd, ...
    ending
        congratulations.psd, ...
    levelselect
        clickhere.psd, ...
    mainmenu
        welcome.psd, ...
    etc

A simplified but functional version of the script we use is this:

#target photoshop

var target = "/C/projects/ds/images";
var toLocalize = new Array();
var totalProcessed = 0;
var warningsFiles = new Array();

function log(msg) {
	$.writeln(msg);
}

function processPSDFolder(dir) {
	var files = dir.getFiles("*.psd");
    log("===============================================");
    log(dir + " -> "+ files.length);
    log("===============================================");
	for (var i = 0; i < files.length; i++) {
    	var doc = app.open(files[i]);
    	log("  file: " + files[i]);
        totalProcessed++;
        if ( doc.artLayers.length == 0 ) {
            log("   -> WARNING: ZERO TEXT LAYERS? THEY SHOULD NOT BE IN GROUPS.");
            warningsFiles.push(files[i]);
        }
    	for (var j = 0; j < doc.artLayers.length; j++) {
        	var lyr = doc.artLayers[j];
        	if (lyr.kind == LayerKind.TEXT) {
            	var lyr = doc.artLayers[j];
            	log("   ->" + lyr.textItem.contents);
            	toLocalize.push(lyr.textItem.contents);
        	}
     	}
    	doc.close(SaveOptions.DONOTSAVECHANGES);
	}
}

function saveStrings() {
	var out = new File(target + "/strings.txt");
	out.open("w");
	for (var i = 0; i < toLocalize.length; i++) {
		var str = toLocalize[i];
		out.writeln(str);
	}
 	out.close();
}

var root = Folder(target).getFiles();
for ( var i = 0; i < root.length; i++ ) {
	var fileFoldObj = root[i];
	if ( fileFoldObj instanceof File ) {         
        // Discard files at this level
	} else {
         processPSDFolder( Folder(fileFoldObj) );
	}
}
saveStrings();
log("Total PSDs processed: " + totalProcessed);
log("Warnings: " + warningsFiles.length);
for ( var i = 0; i < warningsFiles.length; i++ ) {
	log(" " + warningsFiles[i]);
}

Observations:

  • target is the path to your directory structure holding the PSD files.
  • The strings to be translated will we written to file strings.txt under your target directory.
  • This script looks for text layers on the top level of the PSD. It can be easily extended to inspect layers in groups, though.
  • We use warnings to be notified about files not containing Text Layers. These might be files requiring special exporting and extra formatting, and therefore we’ll have to handle such files exceptionally.