Linux4Palm Development Workstation on Gentoo Linux

The title says it all ! :-)

Kiel oni skribas ... ? Typing Esperanto characters in Ubuntu

[update 20081102: Finally, almost three years after the bug report, Ubuntu 8.10 Intrepid Ibex has full, built-in support for Esperanto letters. This post may be considered out-of-date now. Congratulations for the gtk+ team !]

Gtk+ (and thus Ubuntu) multilingualism environment is not as good as it could be. Imagine that you can write in some exotic language. Now try to write a blog in such language using firefox in Ubuntu. You'll hit a wall and that's it.

For the impatient: add the line:

GTK_IM_MODULE=xim

In your /etc/environment file and reboot. [update 20080317: gtk will be fixed so that this is not necessary anymore, see gnome bug 321896] For the patient: read on.

Interestingly, a set of small bugs, still present in Feisty Fawn, makes me unable to write in Esperanto, an interesting constructed European language (that I learn just for fun), that uses the following special characters: ĉ, ĝ, ĥ, ĵ, ŝ and ŭ. The first, and most basic problem is the following: gtk+ could allow, with zero configuration, anybody write such characters (on a typical international keyboard). Qt based and Xlib based application allow this. If I used Kubuntu I wouldn't be writing this post, because Esperanto characters would work out of the box !

You may try this out in your Ubuntu or gnome based distribution: in firefox and gedit, type ^ + a, ^ + e (using an international keyboard). You'll have the â and ê characters. Now type ^ + c, ^ + h. Firefox and gedit will beep and won't show neither ĉ nor ĥ. Now open xterm and type ^ + a, ^ + e, ^ + c, ^ + h. You'll have âêĉĥ.

In X-Windows, complex character typing is handled by modules called input methods. X has a default API for handling input methods, called XIM. That's what xterm uses. Gtk+ provides it's own API and set of input methods for gtk+ based applications. Gtk+ input methods don't provide Esperanto characters. Why ? Simply because gtk+ developers don't want to. I have filed this bug more than one year ago. Also, I could find a similar bug in gnome database that is 4,5 years ago. Now, in your Ubuntu, install some Qt/KDE application and type âêĉĥ. Easy, ain't it ?

Fortunatelly, one of the input methods from the gtk+ set is to use the fallback XIM input method. It's the input method that says "I was unable to do what I was supposed to, so I'll let you use the standard, simpler input method because it works". In gedit, right-click in the text area, select the menu "input methods", then "X input method". Now type âêĉĥ. Voilà ! Firefox doesn't have this menu, however :'(. How one could make the X input method the default method, so that all applications work and I don't have to select it from a menu ? Here we go to our second multilingualization bug. There's no way, in gnome or gtk, to set our default input method. That's right. No way. Ni feliĉe batu en la muron, denove !

The solution is to edit by hand the configuration file given above, /etc/environment (or in your personal enviroments file, ~/.bashrc, or maybe in some other place, depending on your system). In the next boot, when applications load the new environment, XIM will take control. Finally, we have a multilingual environment for Ubuntu... but wait !

Firefox still doesn't work. Why is that ? Here we have a Ubuntu specific bug. More specifically: a 64-bit Ubuntu bug. Almost all the applications I use are compiled for 64-bit. Firefox (and a few others) is an exception. I need to browse sites that use Java and Flash software. Java has just become free software. Flash is closed source software. Neither of them offer 64-bit support for firefox. So my firefox must be 32 bits (see Ubuntu forums if you'd like to know more about this). There are a few support packages that I have to install in order to support 32 bit applications, like "ia32-libs", "ia32-libs-gtk", etc. Those applications don't correctly install the 32 bit environment. Two extra hacks are necessary for 32 bit, gtk apps, to work. The first: add the following line to your firefox32 script (mine is /usr/local/bin/firefox32 or /usr/local/firefox32/firefox):

GTK_IM_MODULE_FILE=/etc/gtk-2.0/gtk.immodules.32

This first bug means that the 32 bit gtk looks by default in gtk.immodules, instead of gtk.immodules.32. The second bug is that the base 32 bit environment doesn't set locales. You need the following command:

ln -s /usr/lib/locale /usr/lib32/locale

This should do. Notice how we pursued a single task: to write in Esperanto in Ubuntu. We've found 4 bugs. Not the way to go.

Beginning Ruby on Rails

For those like me who may be wondering if web development is cool, the place to start is Ruby on Rails. Rails is a much famous web application framework, valued for its agility. In Rails home page, there's an interesting video of someone creating a complete web site in 15 minutes. An exciting tutorial is available in ONLamp.com web site, "Rolling with Ruby on Rails Revisited". Unfortunatelly I haven't found (as of April 2007) any free comprehensive manual of Rails on-line (there's a good oportunity here, eh ? :-) ), so I've decided to buy the book (Agile web development with Rails).

PS: for those who want to stick with python, an alternative to Rails is available, and it's called TurboGears ;-).

New GPIO API in linux kernel 2.6.21 !!!

The 2.6.1 kernel now features GPIO documentation and API !!! :-) See Documentation/gpio.txt.

A case for consistent (and beautiful) coding styles

As it's written in The Zen of Python (PEP 20), "Readability counts". Code is read much more often than written, so a nice looking source code is fundamental.

This is a criticism specifically aimed to the inconSistentCase that started to plague C++, C and even assembly code in the Internet, and, more generally, a criticism of ugly, useless coding styles, like pre/post-fixed notations, very long line styles, and, surely, inconSistentCase.

Whenever you write code or are in charge of deciding the coding style of your software, please, please, please, use consistent coding styles. For example, use full, first-letter-caps, CamelCase notation, not inconSistentCase. As for my personal coding style taste, do even more, minimize the amount of capitalized letters in your code and use the highly legible underscore_notation instead, like those used in most C code, in python and, especially, in C++ STL. Don't let your code strech to hundreds of characters per line and don't add completely useless "NrUserId" prefixed notations.

Here are many examples of nice, well thought coding styles, that you may adapt for your needs:

• PEP 8: an exellent coding style guide for the python language. Mostly an underscore_notation style, with a little, easy on eye, CamelCase and post-fixed notation for special cases (ex: ClassNames, ExceptionsEndInError, etc.). The style is good for python and other high level languages, but it could be better for C++, which leads to...
• STL: part of the C++ standard library, a good example of really well though piece of code. Uses full underscore_notation. If paired with selective namespace downloading (using std::vector, using std::string, etc.), will produce beautiful C++ code.
• Xlib: Choose this one if you prefer a mostly CamelCase style. Even though Xlib is outdated and supposed to fade away, it was also a well thought library. It also shows a good way to add a namespace-like structure to a C library, using a single "X" prefix in most library functions.
• linux kernel: Another great coding style for the C language. Useful for mostly non-object oriented, structured programming, but includes insightful hints for anybody (ex: local variables get short, "aux, tmp" names, external functions get long_descriptive_names, etc.), and is backed by the pseudo-chaotic-multi-million-people linux kernel development success.

There are many other projects that you may try that use similar, high legibility coding standards, just google for it. See, for example, Gtkmm, Gtk+ port to C++, the OpenTTD game, etc.

If you've found that all those coding styles are crappy (even after thinking of them in terms of readability), then you should just remember one thing from this post: don't use inconSistentCase, it sucks.

Logged in Palm TX ! :-)

Amazing ! Logged in Palm TX !

Since Palm TX linux USB support in hackndev kernel works flawlessly, I was able to deploy buildroot's chroot jail and setup sshd in it ! I'll post a screenshot soon. :-)

Linux on Palm TX !

This week, I bought a Palm TX, from a friend of a friend. I've been able to boot linux on it, using hackndev howto. I hope that I'll be able to research more fronts now (kernel, applications, performance, etc.), because Palm TX is expected to have a working USB support (so that I can telnet on it :-)) and because of the different capabilities of Palm Tungsten E2 (PXA 255, 16 MB main memory, 320x320 + input area) and Palm TX (PXA 270, 32 MB main memory, 320x480, no input area).

Reference on GPIO programming with Intel PXA

In my quest to make the USB controller to work without messing the rest of the system, I've found an interesting reference about GPIO configuration for linux ARM. Devices may be activated/deactivated with GPIO ports and, as described in TE2 forum thread at hackndev, USB requires controling GPIO 53.

Kernel (almost) correctly recompiled for Tungsten E2 !!!

Good news ! I was able to boot a home compiled kernel for Palm Tungsten E2 today. :-) This time, I downloaded an older kernel revision from hackndev svn (revision 382, I think, from April 2006). It's a modified 2.6.16 and it's probably the revision nerdfeliz used to compile his kernel. The adjustments were simple, and after some trial and error, I was able to setup an USB serial port. Here are the steps to build the TE2 kernel:

• download hackndev kernel revision 382
• in the main Makefile, change the CROSS_COMPILE to reflect the prefix name of your cross toolchain (mine was "arm-linux-")
• in arch/arm/mach-pxa/palmte2/palmte2_buttons.c, remove the __exit macro from palmte2_buttons_remove
• do a make menuconfig and load nerdfeliz's configuration 20060406.
• add usb gadget support, pxa usb support and serial gadget with CDC ADM support
• in boot options, add "g_serial.use_acm=1" option to change usb serial behavior
• make and cp
  

That's it. This kernel should boot. :-) To activate USB, we need the following trick:

• mknod /dev/usb_serial c 127 0 (in TE2 /dev directory, not in your PC ;-) )
• add a "getty -L 38400 usb_serial" line in TE2 /etc/inittab
• load Garux and set the "Open Cradle USB" option
• boot with the device connected to the PC
  
• after the many usb error messages disconnect the USB cable, wait your desktop flush /dev (wait 5 seconds) and plug the USB connector again :-D
• open gtkterm or minicom and open /dev/ttyACM0 in your desktop (or ACMx if you have other ACM devices)

All right, then ! Now, we have a working ARM linux system and a really expensive dumb terminal linked to it !!! Not quite. :-( I was unable to log in the remote system using gtkterm or minicom, either with "root" user or "default" (buildroot default user), even after playing with securetty and passwd/shadow configurations. Also, there's another sad bug: there's some kind of problem with the SD card driver, it keeps popping error messages. More to come...

Getting the vsyscall address from the ELF auxiliary vector [ref]

In a previous post I have showed how to make new style system calls in linux x86. I've also remarked that assuming that the __kernel_vsyscall address was always at a fixed address was wrong. The proper way to get the function address would be declare it as "extern", and link it with with linux-gate.so.1. However, I don't know how to do that, because linux gate is virtual and ld can't find it. I'll post here another way, that gets the information passed to the running software by the ELF loader. It retrieves an auxiliary ELF structure, that is present in the stack, after the usual argc, argv and envp pointers.

First, let's modify our previous hello.asm to call a new function "get_kernel_vsyscall", and set the address returned as the system call address:

; hello.asm
bits 32
global _start
extern get_kernel_vsyscall ; new function in vsyscall.asm
section .data
     hello: db 'Hello World !', 10
     hello_len: equ $-hello
section .bss
     linux_gate: resd 1 ; now, linux gate is variable
section .text
_start:
     ; New code to save the vsyscall address
     mov eax, esp
     call get_kernel_vsyscall
     mov [linux_gate], eax

     mov eax, 4 ; write()
     mov ebx, 1
     mov ecx, hello
     mov edx, hello_len
     call [linux_gate] ; call with dereference

     mov eax, 1 ; _exit()
     mov ebx, 42
     call [linux_gate] ; call with dereference


So, we changed "linux_gate" into a variable and filled it with the address returned by "get_kernel_vsyscall", basically. This function loops in the stack, beginning with argc, until it finds the correct (key, value) pair in the ELF header:

; vsyscall.asm
bits 32
global get_kernel_vsyscall
%define ELF_VSYSCALL_ID 32 ; defined in elf.h as AT_SYSINFO
%define ELF_EOF_ID 0 ; defined in elf.h as AT_NULL

section .text
get_kernel_vsyscall:
; Returns the kernel vsyscall address from the
; ELF aux vector. The vsyscall address is the
; subroutine used for the new style system call.
; in:
;     eax: the address, in the stack, of "argc"
; out:
;     eax: the kernel vsyscall address, or zero
;     on error

     push ebx

     ; First we skip argc and argv
     mov ebx, [eax]
     inc ebx
     inc ebx
     shl ebx, 2
     add eax, ebx

     ; For envp, we need a loop until NULL is found
envp_loop:
     mov ebx, [eax]
     add eax, 4
     cmp ebx, 0
     jne envp_loop

     ; Now we are in the ELF aux vector. Look for
     ; the correct (key, value) pair
     mov ebx, [eax]
     cmp ebx, ELF_VSYSCALL_ID
     je vsyscall_id_found
elf_vector_loop:
     add eax, 8
     mov ebx, [eax]
     cmp ebx, ELF_VSYSCALL_ID
     je vsyscall_id_found
     cmp ebx, ELF_EOF_ID
     jne elf_vector_loop
vsyscall_id_found:
     add eax, 4
     mov eax, [eax]

     pop ebx
     ret

Weird, eh ? We needed to look for the AT_SYSINFO field (ELF_VSYSCALL_ID in the code) in some extra vector provided by the ELF loader. Other executable formats would give those parameters some other way. We have two options, now: use a hardcoded vsyscall address or retrieve it from the ELF vector. Compile and link with "nasm -f elf hello.asm; nasm -f elf vsyscall.asm; ld -o hello hello.o vsyscall.o".

Hackndev custom linux kernel for PDAs [ref]

I've downloaded hackndev modified linux kernel. It's a heavily modified kernel, with support for a lot of ARM PDAs. After reading TE2 mailing lists and some fixups I was able to compile a Tungsten E2 specific kernel. I couldn't boot it, however, the screen just faded after booting and nothing else. I tried two different kernel revisions and two different "config" files, with no success. The kernel image that is correctly booting in my Palm is from April 2006. I'll continue my research to a proper kernel in hackndev svn around this date.

New style system call in linux x86 [ref]

We all know that the x86 architecture is bizarre. An interesting event that has happened in the last 10 years was the change of the "system call" instruction. We all remember from DOS time that we issued "int 0x21" to make a DOS system call, and "int 0x10" to make a ROM BIOS system call. Some of us has even played with the linux system call, "int 0x80". However, five years ago, when using a Pentium IV, someone has found out that the "int" instruction could be half as fast in a Pentium IV than in a Pentium III. Why was that ?

Intel introduced a new system call instruction in Pentium Pro/Pentium II, called "sysenter" that was supposed to be a "Fast System Call" facility. This meant that Pentium processors had two system call instructions: one that was the fast system call and the other, the slow system call (currently, there are three system calls: "int", "sysenter" and "syscall" !!!). Linux has since migrated to new style system calls. Instead of changing the "int" logic to the "sysenter" logic, another approach was taken. A new virtual shared library was created, called "linux gate". This library is virtual because it's like a standard .so library, but only exists in memory (do a "ldd /bin/ls" and see if your system is using "linux-gate.so.1"). Applications (like libc or other system call front-ends) may then call a simple function exported from this library, __kernel_vsyscall, and this function will "systenter", "int" or issue whatever instruction is necessary for completing the system call. So here we have quick dirty "hello world" written in NASM assembly for linux (should work on linux 2.6 for x86). The code assumes that __kernel_vsyscall is always mapped at 0xffffe400 in memory (which is wrong ;-)).

; hello.asm
bits 32
global _start
section .data
   linux_gate: equ 0xffffe400
   hello: db 'Hello World !', 10
   hello_len: equ $-hello
section .text
_start:
   mov eax, 4 ; write()
   mov ebx, 1
   mov ecx, hello
   mov edx, hello_len
   call linux_gate

   mov eax, 1 ; _exit()
   mov ebx, 42
   call linux_gate

Notice that we are making a procedure call, not a system call. Compile it with "nasm -f elf hello.asm; ld -o hello hello.o" . :-)

Pythonic C++

Pythonic C++ may not be possible. :-) However, this simple trick will allow you to better follow PEP 8 without recurring to pre/post-fixed notations when accessing variable or function members:


Car::Car(unsigned engines) :
  Vehicle(4, engines)
{
  // use reference instead of pointer ;-)
  Car &self = *this;

  self.position = 5;
  self.movement = 10;
}

Amazing, isn't it ? :-D With just a simple trick we have boosted code readability by one million. PS: how do I highlight "self" ? :-)

Back to buildroot, successful root system deployed

I was unable to compile glibc with the target architecture set to xscale-linux-elf. I think I should have used arm-linux at this moment, which is the same that buildroot uses. Back to buildroot, I've removed the software that was giving compile errors (jffs2/mtd utils) and was able to create a basic root filesystem. I was unable to use the system, however, because I don't have an installed keyboard and there was no virtual keyboard installed. I was unable to boot a newly compiled ARM stock kernel, with USB gadget support for telneting into it. Now I'll rebuild buildroot with Tiny X and Qtopia while I download hackndev's modified kernel and try to enable USB support.

Cross compiling binutils [ref]

After being unable to automate a Xscale toolchain cross-build for my Palm TE2 with buildroot, I'll use the MIPS Cross Linux From Scratch to derive a native toolchain for TE2. The binutils cross-build worked with CLFS_TARGET=xscale-linux-elf.

Linux on Palm Tungsten E2 How-to [ref]

Meet Garux, a linux bootloader for Palm OS. There's a specific Palm Tungsten E2 Garux build in that page, along with instructions on how to install Opie on it.