16 Jan

16 bits COM Oddity

I can’t even pinpoint what a 16 bits COM Oddity really means, but I think the idea is therein, somehow. Previously, I explained how to code a simple a “hello, world” program using the DEBUG tool that was shipped with DOS. Revisiting this obsolete knowledge was unexpectedly fun. We’ll retrieve the hexadecimal version of “hello, world” (well, “hello, world!!”) from that post:

EB 13 0D 0A 68 65 6C 6C 6F 2C 20 77 6F 72 6C 64
21 21 0D 0A 24 B4 09 BA 02 01 CD 21 B4 00 CD 21

That’s all we need for our “hello, world!!” binary. 32 bytes exactly. We can create that file bit by bit but that’d be too excessive, I think. Let’s use the echo command instead. This is the full command I entered in my Windows 10 cmd.exe prompt:

echo|set /p="Ù‼♪◙hello, world!!♪◙$┤○║☻☺═!1└═!">hello.com

After that you’ll get a 16-bit COM, hello.com, that will display the “hello, world!!” message. Funny 🙂

What are those weird characters?

First a little explanation. We want our hello.com file to be, byte after byte, an exact representation of the hexadecimal sequence above presented. We’ll use cmd.exe commands to dump characters into the file and, if we choose our characters carefully in order to match the target hexadecimal values, we’ll end up with the exact representation we’re looking for. For instance, the first 2 bytes block, EB 13, is the “jmp 115” instruction. Then comes the newline (0D 0A), and so on. If we convert our hexadecimal to decimal, we get:

235 19 13 10 104 101 108 108 111 44 32 119 111 114 108 100 
 33 33 13 10  36 180   9 186   2  1 205 33 180   0 205  33

The first byte in hello.com must be EB, or 235 in decimal. In order to dump our characters from the command line, we’ll convert that decimal value to a character. I’m trying this on a Windows 10 (64-bits) machine, with cmd.exe using Code page 850 Multilingual Latin 1. In such code page, character 235 is Ù. And 19 is ‼. And, luckily, 13 is ♪ and 10 is ◙. Those two characters are especially important because they represent the carriage return and the line feed, respectively, and some shells won’t convert them to characters. However, happily, cmd.exe with my default code page will handle them as we need. To input those characters you can type the usual ALT + decimal value.

There are a few important things to notice:

Read More
15 Jan

“Hello world” with DEBUG

Coding “Hello world” with DEBUG will be a blunt exercise on programming futility. Or an exercise on retro, old-school coding. More than two decades ago I used to code in x86 (Intel) assembly, almost daily. I remember the masochist approach to learning the opcodes and the hardware architecture. The famous RBIL (Ralf Brown’s Interrupt List) was, back then, my favorite “reference”. First painful steps were taken and first crashes happily followed. I remember trying to code, as expected, the traditional “hello, world!”, using a strange tool included in DOS, DEBUG.COM. I wrote a post about this “hello, world” with DEBUG.COM elsewhere, and yesterday I found the time to reread it: I verified, first with awe, then with horror, and finally, with relief, that I had almost completely forgotten how to code in assembly. So I’ll revisit this here, mostly as a self-imposed disciplinary measure, an exercise on programming, specifically, an exercise on programming futility. Heck, DEBUG isn’t even available on the Windows 10 machine I’m typing this on. However, DEBUG looked pretty cool back then: it could assemble, disassemble and dump hexadecimal output. You could create little programs, or inspect programs and peek memory areas.

Specifically what I want is to build a minimal “hello, world!” program using DEBUG.COM. I don’t have any use for this, but it comes as a “relaxing” post after several weeks focused on the release of “DragonScales 3: Eternal Prophecy of Darkness” on Steam and the localization of “DragonScales 5: The Frozen Tomb”. After we execute DEBUG.COM we’ll meet a prompt with a “-” symbol. Now we can input our commands. I want to assemble, i.e., I want to type assembly language instructions. The command for that is “a”, which might be optionally followed by a memory address. By default, instructions will be placed starting from CS:0100, so I’ll use that address. Equivalently, I could type “a 0100” or “a 100” to achieve the same result.

-a

Now we have to place the data in memory. For this little program I only need the characters for “hello, world!!”. Notice I want two “!!” at the end. That’s because I want the final program to occupy exactly 32 bytes; we’ll see the reason for this later on. I’ll use the pseudo-instruction “DB” to define our string. With DB I can neatly provide the string using ASCII values, like this:

 db "hello, world!!"

Those are 14 bytes. However, I want a prettier output, with a newline character before and after our string. A newline is in fact two characters: a carriage return (CR is ASCII 13) and a line feed (LF is ASCII 10). In hexadecimal, CR is 0Dh, and LF is 0Ah. OK. Now our DB would be modified to look like this:

 db 0d,0a,"hello, world",0d,0a

Those are 18 bytes. We are not done yet with our data. In order to effectively print the message to the standard output I’ll recur to the function 09h of INT 21h. Check RBIL D-2109. In short, I have to place the value 09h in register AH, and DS:DX should point to the beginning of our string. The function will print every character until finding a “$” character (i.e., “$” acts as the “zero” in null-terminated C strings). ASCII value of “$” is 36, or 24 in hexadecimal. Therefore, we modify our DB instruction again:

 db 0d,0a,"hello, world",0d,0a,"$"
Read More
08 Dec

Inversion of Control in Software Engineering

What is Inversion of Control in Software Engineering?

Inversion of Control in Software Engineering (IoC) is essentially a strategy for decoupling software components. Basically, the purpose of IoC is to remove unnecessary dependencies (couplings) which might reduce flexibility and elegance of a system’s design. The term Inversion of Control in Software Engineering might sometimes appear applied to different design contexts. It is a term that comes from the 70s, when bottom-up parsing was considered an inversion of top-down parsing. It has roughly continued to be applied in multiple contexts. Some people think Inversion of Control in Software Engineering is a design strategy. For others, it’s a design pattern. After some articles dealing with OpenGL, in this post we will briefly discuss this topic of Software Engineering.

Read More
04 Dec

Render a Triangle with OpenGL

This post will discuss how to render a triangle with OpenGL. In the following, renderization of a triangle assumes modern OpenGL, i.e., the old, fixed-function pipeline is of no concern for us in this post, as we’ll be using OpenGL buffer objects and shaders.

A Simple Triangle

By following the tutorials in the previous posts (Setting up Eclipse CDT for OpenGL and the GLFW Example) we were able to create a minimal program displaying an empty window. Now we want to draw something with OpenGL on that window, specifically, a triangle. Why a triangle? Well, the geometric shape more frequently used to approximate surfaces is the triangle. Approximation of 3D surfaces in real-time graphics by means of simpler shapes is known as tessellation. For our tutorial purposes, a single triangle will suffice.

GPU Power

Modern GPUs are quite fast and can also have a considerable amount of dedicated memory. When rendering, we’d like for as much rendering data as possible to be read by the GPU directly from its local memory. In order to render a triangle with OpenGL we’ll need, obviously, to transfer the 3 vertices of the triangle to the GPU’s memory. However, we do NOT want our rendering to go like this:

  • read a vertex from our computer RAM
  • copy it to the GPU memory
  • let the GPU process that single vertex
  • and then repeat this whole process for the next vertex of the triangle.

Ideally, what we want is to transfer a batch of data to the GPU’s memory, copying all the triangle vertices, and then letting the GPU operate with this data directly from its local memory. In OpenGL we have the concept of Vertex Buffer Object (VBO) to represent these data placed on GPU’s memory.

The data to render the triangle in OpenGL

Normally, we think of a vertex as a point, which in 3D space leads to a representation with 3 coordinates, commonly designated by x, y and z. However, in this case I’d like to think of a vertex as a more abstract concept: a minimal data structure required to define a shape. Given a vertex, we can “link” attributes to it to further define our shape. Thereby, one of such attributes of a vertex can be its position (the “x, y, z values”.) Other attribute might be the vertex’s color. And so on. In this tutorial we will “link” two attributes to our vertices: position and color. For position we will have three coordinates, each a floating point value. If a float takes 4 bytes, then our position attribute would require 3 x 4 = 12 bytes. For the color attribute, we’d have 3 extra components, following the RGB model. Each color component would then take 4 bytes, and the color attribute would also require 12 bytes. In total, each vertex would take 24 bytes, 12 for its position attribute, and 12 for its color attribute.

Now we have to specify how to process these vertices.

Read More
03 Dec

GLFW Example

Here I’ll briefly discuss a tiny GLFW example. Previously, I explained how to setup Eclipse CDT to work with OpenGL, using GLFW and GLAD. However, I instructed to copy-paste the example code on GLFW Documentation page, without providing any details. In the following I’ll present some code that you can add to the little project of our setup post, and will include GLAD initialization too.

Read More
27 Nov

Setting up Eclipse CDT for OpenGL with GLFW and GLAD

What’s OpenGL?

OpenGL is an API to render 2D and 3D graphics. Remember that an API (Application Programming Interface) is an interface for interaction between components of a system. Typically, an API defines a set of functions, protocols and/or tools. I’ll skip the details about the client-server model, but OpenGL allows a client program to communicate with GPUs (Graphic Processing Units, e.g., your videocard) to achieve faster, hardware-accelerated rendering. That’s why OpenGL is a common topic in the game development scene.

OpenGL is focused on just rendering. It’s an API to write and read data from a framebuffer, and that’s it. It won’t handle user input, or sound playback, or loading a PNG image. It does not even have functions to create or close a window. We’ll need auxiliar libraries for all of that.

A minimal OpenGL window

So we want to build a minimal OpenGL application on Windows. We’ll create an empty window with an OpenGL context, using the GLFW and GLAD libraries. In the following, I assume we’re using a 64 bits version of Windows. I’ll also be relying on mingw-w64. In summary, these are our assumptions:

  • Windows operating system (64 bits.) Things will be a tad different for macOS and Linux users.
  • Eclipse CDT.
  • mingw-w64 to build GLFW from sources. Besides, our Eclipse CDT project will be compiled with the gcc version of mingw-w64.
  • GLFW and GLAD libraries.

What’s GLFW?

As told, OpenGL does not provide any facility to create a window, retrieve user input, create the OpenGL context, etc. These functionalities depend on the operating system. GLFW is a C library which provides a neat abstraction layer to handle all of this on several platforms. Notice that GLFW is focused on management of windows, OpenGL contexts, user input and time. It will not play sounds, or load images, etc.

Read More
25 Sep

UTF-8 with BOM

When localizing DragonScales 3 we experienced a baffling issue with an internal tool whose purpose is simply to replace text in a group of files. Those UTF-8 encoded files contain messages loaded by the game from the very beginning. However, after running the tool, the game started crashing when reading such files. By using an old buddy, fc /B, we found out that our tool was “injecting” a few extra bytes at the start of the file: EF BB BF. In short, the tool was altering the encoding of files from UTF-8 to UTF-8 with BOM. That was the cause for the crashing, as our game expects the files to be UTF-8 encoded without BOM.

What’s this BOM, anyway? Simply put, it’s just a sequence of bytes (EF BB BF) used to signal readers about the file being UTF-8 encoded. It seems such mark might be useful in some specific contexts, with some specific programs. Not our case, so we had to remove the BOM with a little batch script like this:

for /r ".\DE\scenes" %%i in (*.*) do (
  copy %%i .\tmp.txt /Y
  sed -i '1s/^\xEF\xBB\xBF//' .\tmp.txt
  attrib -R .\tmp.txt
  move /Y .\tmp.txt %%i
)

In this snippet we remove the BOM via sed. Files are those under a fictitious directory, .\DE\scenes. Those copies and attribs help to circumvent some problems with permissions of files created by our sed version on Windows.

31 Aug

Extracting text from PSD files

Typically, we need to translate hundreds of strings when localizing our games. Most strings are text messages which the game loads from some database or simple text file. However, we often have to handle localization of several PNG images, such as the one below.

AWESOME! message in DragonScales

Such PNG images are exported from PSD files which must obviously contain at least one Text Layer. To speed up the localization process we have a little Photoshop script which opens the PSD files and extracts all the text we have to translate. PSD files are grouped in directories corresponding to the tileset they belong to. For instance, this would be a typical directory structure for the DragonScales games:

images/
    board/
        awesome.psd, great.psd, ...
    cards
        youwin.psd, sorry.psd, ...
    ending
        congratulations.psd, ...
    levelselect
        clickhere.psd, ...
    mainmenu
        welcome.psd, ...
    etc

A simplified but functional version of the script we use is this:

#target photoshop

var target = "/C/projects/ds/images";
var toLocalize = new Array();
var totalProcessed = 0;
var warningsFiles = new Array();

function log(msg) {
	$.writeln(msg);
}

function processPSDFolder(dir) {
	var files = dir.getFiles("*.psd");
    log("===============================================");
    log(dir + " -> "+ files.length);
    log("===============================================");
	for (var i = 0; i < files.length; i++) {
    	var doc = app.open(files[i]);
    	log("  file: " + files[i]);
        totalProcessed++;
        if ( doc.artLayers.length == 0 ) {
            log("   -> WARNING: ZERO TEXT LAYERS? THEY SHOULD NOT BE IN GROUPS.");
            warningsFiles.push(files[i]);
        }
    	for (var j = 0; j < doc.artLayers.length; j++) {
        	var lyr = doc.artLayers[j];
        	if (lyr.kind == LayerKind.TEXT) {
            	var lyr = doc.artLayers[j];
            	log("   ->" + lyr.textItem.contents);
            	toLocalize.push(lyr.textItem.contents);
        	}
     	}
    	doc.close(SaveOptions.DONOTSAVECHANGES);
	}
}

function saveStrings() {
	var out = new File(target + "/strings.txt");
	out.open("w");
	for (var i = 0; i < toLocalize.length; i++) {
		var str = toLocalize[i];
		out.writeln(str);
	}
 	out.close();
}

var root = Folder(target).getFiles();
for ( var i = 0; i < root.length; i++ ) {
	var fileFoldObj = root[i];
	if ( fileFoldObj instanceof File ) {         
        // Discard files at this level
	} else {
         processPSDFolder( Folder(fileFoldObj) );
	}
}
saveStrings();
log("Total PSDs processed: " + totalProcessed);
log("Warnings: " + warningsFiles.length);
for ( var i = 0; i < warningsFiles.length; i++ ) {
	log(" " + warningsFiles[i]);
}

Observations:

  • target is the path to your directory structure holding the PSD files.
  • The strings to be translated will we written to file strings.txt under your target directory.
  • This script looks for text layers on the top level of the PSD. It can be easily extended to inspect layers in groups, though.
  • We use warnings to be notified about files not containing Text Layers. These might be files requiring special exporting and extra formatting, and therefore we’ll have to handle such files exceptionally.
24 Feb

Comments about warning “Deprecated Carbon Component Manager” on OS X El Capitan

Last November, after updating to OS X “El Capitan”, we started seeing a strange warning message when running DragonScales 2:

“WARNING: 140: This application, or a library it uses, is using the deprecated Carbon Component Manager for hosting Audio Units. Support for this will be removed in a future release. Also, this makes the host incompatible with version 3 audio units. Please transition to the API’s in AudioComponent.h.”

DragonScales 2 was built with LibGDX, and before updating to “El Capitan” it did run with no problems, warnings, etc. After some research, we were informed that this was caused by an OpenAL-Soft issue (which has already been fixed.) Specifically, Apple is deprecating some libraries, e.g., the Carbon Component Manager, and the OpenAL-Soft library was referencing such deprecated Carbon Component. When a game referencing these deprecated libraries is executed, newer OS X (e.g., El Capitan) shows the above warning. However, as told, this OpenAL-Soft issue was already solved on last November: the library is updated and ready for prime time.

A build of OpenAL-Soft is part of the LWJGL natives bundled with LibGDX. If you don’t use an updated build of OpenAL-Soft you’ll keep receiving the deprecation warning. As we commented on this thread, a quick fix is downloading the latest LWJGL3 build, grab the native libopenal.dylib and drop it over the OpenAL native bundled with LibGDX.

A caveat, though, about using the OpenAL-Soft included in LWJGL3 (at time of writing.) Some testers of DragonScales 2 for OS X reported a nasty crash. Here’s part of the crash report:

Dyld Error Message:
Symbol not found: ___sincosf_stret
Referenced from: /var/folders/*/openal.dylib
Expected in: /usr/lib/libSystem.B.dylib

Clearly, our OpenAL library was referencing a function not provided by OS X. It turns out that our testers were using an older OS X version (10.7, I think) whereas the libopenal.dylib bundled with LWJGL3 was targeting OS X 10.9. In fact, output of otool -l libopenal.1.17.0.dylib includes these lines:

cmd LC_VERSION_MIN_MACOSX
cmdsize 16
version 10.9
sdk 10.9

As our publishers require support for OS X >= 10.7, we had to compile our own libopenal.dylib. We set OS X 10.6 as deployment target and 10.11 (El Capitan) as the root SDK, by using these CMake variables:

-DCMAKE_OSX_DEPLOYMENT_TARGET=10.6 -DCMAKE_OSX_SYSROOT=macosx10.11.

By the way, here you can find OS X SDKs if you need them.

Checking our build with otool, yields:

Load command 8
cmd LC_VERSION_MIN_MACOSX
cmdsize 16
version 10.6
sdk 10.11

This version of the library passed all our tests successfully.

27 Aug

Resizing multiple image files with ImageMagick

For the first DragonScales post I had several PNGs with a 1920×1200 resolution. Nevertheless, the ideal width for images in this blog is about 600 pixels:

DragonScales Logo (Black background)

DragonScales Logo (Black background)

This is a frequent issue when publishing images here. Thereby I recur to ImageMagick in order to resize sets of images. ImageMagick’s command line tools are excellent and indispensable tools for our work. For instance, on Windows command prompt, it’s enough to type:

forfiles /M *.png /C “cmd /c convert -resize 31% @file resized_@file”

That way all of the images will have resized versions, at 31% their original size, yielding a 600-pixels width approximately. Very useful. And ImageMagick is a great piece of software.