Developing CZ, a dialect of C that looks like Python

In my experience, the C programming language is still hard to beat, even 50 years after it was first developed (and I feel the same way about UNIX). When it comes to general-purpose utility, low-level systems programming, performance, and portability (even to tiny embedded systems), I would choose C over most modern or fashionable alternatives. In some cases, it is almost the only choice.

Many developers believe that it is difficult to write secure and reliable software in C, due to its free pointers, the lack of enforced memory integrity, and the lack of automatic memory management; however in my opinion it is possible to overcome these risks with discipline and a more secure system of libraries constructed on top of C and libc. Daniel J. Bernstein and Wietse Venema are two developers who have been able to write highly secure, stable, reliable software in C.

My other favourite language is Python. Although Python has numerous desirable features, my favourite is the light-weight syntax: in Python, block structure is indicated by indentation, and braces and semicolons are not required. Apart from the pleasure and relief of reading and writing such light and clear code, which almost appears to be executable pseudo-code, there are many other benefits. In C or JavaScript, if you omit a trailing brace somewhere in the code, or insert an extra brace somewhere, the compiler may tell you that there is a syntax error at the end of the file. These errors can be annoying to track down, and cannot occur in Python. Python not only looks better, the clear syntax helps to avoid errors.

The obvious disadvantage of Python, and other dynamic interpreted languages, is that most programs run extremely slower than C programs. This limits the scope and generality of Python. No AAA or performance-oriented video game engines are programmed in Python. The language is not suitable for low-level systems programming, such as operating system development, device drivers, filesystems, performance-critical networking servers, or real-time systems.

C is a great all-purpose language, but the code is uglier than Python code. Once upon a time, when I was experimenting with the Plan 9 operating system (which is built on C, but lacks Python), I missed Python’s syntax, so I decided to do something about it and write a little preprocessor for C. This converts from a “Pythonesque” indented syntax to regular C with the braces and semicolons. Having forked a little dialect of my own, I continued from there adding other modules and features (which might have been a mistake, but it has been fun and rewarding).

At first I called this translator Brace, because it added in the braces for me. I now call the language CZ. It sounds like “C-easy”. Ease-of-use for developers (DX) is the primary goal. CZ has all of the features of C, and translates cleanly into C, which is then compiled to machine code as normal (using any C compiler; I didn’t write one); and so CZ has the same features and performance as C, but enjoys a more pleasing syntax.

CZ is now self-hosted, in that the translator is written in the language CZ. I confess that originally I wrote most of it in Perl; I’m proficient at Perl, but I consider it to be a fairly ugly language, and overly complicated.

I intend for CZ’s new syntax to be “optional”, ideally a developer will be able to choose to use the normal C syntax when editing CZ, if they prefer it. For this, I need a tool to convert C back to CZ, which I have not fully implemented yet. I am aware that, in addition to traditionalists, some vision-impaired developers prefer to use braces and semicolons, as screen readers might not clearly indicate indentation. A C to CZ translator would of course also be valuable when porting an existing C program to CZ.

CZ has a number of useful features that are not found in standard C, but I did not go so far as C++, which language has been described as “an octopus made by nailing extra legs onto a dog”. I do not consider C to be a dog, at least not in a negative sense; but I think that C++ is not an improvement over plain C. I am creating CZ because I think that it is possible to improve on C, without losing any of its advantages or making it too complex.

One of the most interesting features I added is a simple syntax for fast, light coroutines. I based this on Simon Tatham’s approach to Coroutines in C, which may seem hacky at first glance, but is very efficient and can work very well in practice. I implemented a very fast web server with very clean code using these coroutines. The cost of switching coroutines with this method is little more than the cost of a function call.

CZ has hygienic macros. The regular cpp (C preprocessor) macros are not hygenic and many people consider them hacky and unsafe to use. My CZ macros are safe, and somewhat more powerful than standard C macros. They can be used to neatly add new program control structures. I have plans to further develop the macro system in interesting ways.

I added automatic prototype and header generation, as I do not like having to repeat myself when copying prototypes to separate header files. I added support for the UNIX #! scripting syntax, and for cached executables, which means that CZ can be used like a scripting language without having to use a separate compile or make command, but the programs are only recompiled when something has been changed.

For CZ, I invented a neat approach to portability without conditional compilation directives. Platform-specific library fragments are automatically included from directories having the name of that platform or platform-category. This can work very well in practice, and helps to avoid the nightmare of conditional compilation, feature detection, and Autotools. Using this method, I was able easily to implement portable interfaces to features such as asynchronous IO multiplexing (aka select / poll).

The CZ library includes flexible error handling wrappers, inspired by W. Richard Stevens’ wrappers in his books on Unix Network Programming. If these wrappers are used, there is no need to check return values for error codes, and this makes the code much safer, as an error cannot accidentally be ignored.

CZ has several major faults, which I intend to correct at some point. Some of the syntax is poorly thought out, and I need to revisit it. I developed a fairly rich library to go with the language, including safer data structures, IO, networking, graphics, and sound. There are many nice features, but my CZ library is more prototype than a finished product, there are major omissions, and some features are misconceived or poorly implemented. The misfeatures should be weeded out for the time-being, or moved to an experimental section of the library.

I think that a good software library should come in two parts, the essential low-level APIs with the minimum necessary functionality, and a rich set of high-level convenience functions built on top of the minimal API. I need to clearly separate these two parts in order to avoid polluting the namespaces with all sorts of nonsense!

CZ is lacking a good modern system of symbol namespaces. I can look to Python for a great example. I need to maintain compatibility with C, and avoid ugly symbol encodings. I think I can come up with something that will alleviate the need to type anything like gtk_window_set_default_size, and yet maintain compatibility with the library in question. I want all the power of C, but it should be easy to use, even for children. It should be as easy as BASIC or Processing, a child should be able to write short graphical demos and the like, without stumbling over tricky syntax or obscure compile errors.

Here is an example of a simple CZ program which plots the Mandelbrot set fractal. I think that the program is fairly clear and easy to understand, although there is still some potential to improve and clarify the code.

#!/usr/local/bin/cz --
use b
use ccomplex

Main:
	num outside = 16, ox = -0.5, oy = 0, r = 1.5
	long i, max_i = 50, rb_i = 30
	space()
	uint32_t *px = pixel()  # CONFIGURE!
	num d = 2*r/h, x0 = ox-d*w_2, y0 = oy+d*h_2
	for(y, 0, h):
		cmplx c = x0 + (y0-d*y)*I
		repeat(w):
			cmplx w = c
			for i=0; i < max_i && cabs(w) < outside; ++i
				w = w*w + c
			*px++ = i < max_i ? rainbow(i*359 / rb_i % 360) : black
			c += d

I wrote a more elaborate variant of this program, which generates images like the one shown below. There are a few tricks used: continuous colouring, rainbow colours, and plotting the logarithm of the iteration count, which makes the plot appear less busy close to the black fractal proper. I sell some T-shirts and other products with these fractal designs online.

An image from the Mandelbrot set, generated by a fairly simple CZ program.

I am interested in graph programming, and have been for three decades since I was a teenager. By graph programming, I mean programming and modelling based on mathematical graphs or diagrams. I avoid the term visual programming, because there is no necessary reason that vision impaired folks could not use a graph programming language; a graph or diagram may be perceived, understood, and manipulated without having to see it.

Mathematics is something that naturally exists, outside time and independent of our universe. We humans discover mathematics, we do not invent or create it. One of my main ideas for graph programming is to represent a mathematical (or software) model in the simplest and most natural way, using relational operators. Elementary mathematics can be reduced to just a few such operators:

+	add, subtract, disjoint union, zero
×	multiply, divide, cartesian product, one
^	power, root, logarithm
◢	sin, cos, sin-1, cos-1, hypot, atan2
δ	differential, integral

a set of minimal relational operators for elementary math

I think that a language and notation based on these few operators (and similar) can be considerably simpler and more expressive than conventional math or programming languages.

CZ is for me a stepping-stone toward this goal of an expressive relational graph language. It is more pleasant for me to develop software tools in CZ than in C or another language.

Thanks for reading. I wrote this article during the process of applying to join Toptal, which appears to be a freelancing portal for top developers; and in response to this article on toptal: After All These Years, the World is Still Powered by C Programming.

My CZ project has been stalled for quite some time. I foolishly became discouraged after receiving some negative feedback. I now know that honest negative feedback should be valued as an opportunity to improve, and I intend to continue the project until it lacks glaring faults, and is useful for other people. If this project or this article interests you, please contact me and let me know. It is much more enjoyable to work on a project when other people are actively interested in it!

Linux, low power / low heat for summer

Sometimes I play browser games including diep.io.  This loads the CPU and GPU, and in this summer weather my laptop gets too hot and heats up the room.

I tried using Chrome with the GPU disabled, but the browser games would still cause the GPU to ramp up to full clock rate. I guess the X server was using the GPU.

google-chrome --disable-gpu   # does not always prevent GPU clocking up

So here’s what I did:

For the NVIDIA GPU, we can force the lowest power mode by adding the following to the “Device” section in /etc/X11/xorg.conf:

# Option "RegistryDwords" "PowerMizerEnable=0x0;"
Option "RegistryDwords" "PowerMizerEnable=0x1; PerfLevelSrc=0x3333; PowerMizerLevel=0x3; PowerMizerDefault=0x3; PowerMizerDefaultAC=0x3"

Unfortunately the “nvidia-settings” tool does not allow this.  It is necessary to restart the X server in order to change this setting.  Just swap which line is commented out.

Given that we are keeping the GPU cool like this, Chrome works better with the GPU enabled not disabled.

For the CPU, setting “scaling_governor=powersave” does not force the lowest power mode, and the CPU still clocks up and gets hot.  But we can set “scaling_max_freq” to stop Linux from raising the clock speed.  I’m using this shell script “cpu_speed“:

#!/bin/bash
cmd=${1-info}
cd /sys/devices/system/cpu
for cpu in cpu[0-9]*; do
 (
 cd $cpu/cpufreq
 case "$cmd" in
 info) echo $cpu `<scaling_cur_freq` `<scaling_min_freq` `<scaling_max_freq`
 ;;
 slow) cat cpuinfo_min_freq >scaling_min_freq
 cat cpuinfo_min_freq >scaling_max_freq
 ;;
 fast) cat cpuinfo_min_freq >scaling_min_freq
 cat cpuinfo_max_freq >scaling_max_freq
 ;;
 esac
 )
done

I can run it with “cpu_speed” to see the current speed, “cpu_speed slow” to fix the clock at the lowest speed, and “cpu_speed fast” to allow the clock to go up to the maximum speed.

This “temperature” script shows the NVIDIA GPUCurrentPerfLevel, GPUCoreTemp, and CPU temperature info:

#!/bin/sh
(
set -a
: ${DISPLAY:=:0.0}
nvidia-settings -q GPUCurrentPerfLevel -q GPUCoreTemp
acpi -t
) 2>/dev/null |
perl -ne 'print "$1 " if /[:,] (\d+)\./'
echo

Finally, I can reduce the screen resolution to decrease the load on the GPU and CPU.  “xrandr” with the NVIDIA driver does not allow me to change the resolution directly, but there is an option to scale the display.  This gives much smoother performance in the browser games, and the lower resolution doesn’t hurt.

xscale-half:

xrandr --output DP-2 --scale 0.5x0.5

xscale-normal:

xrandr --output DP-2 --scale 1x1

Anyway, now I have my laptop set up to run cool by default.  This doesn’t hurt for most things I am doing with it, and I feel it’s less likely to explode and burn down our house.

Chrome memory abuse, and “swap space” rant

I like the Chrome browser, but the memory usage is fscking ridiculous.

I have a slightly older computer at work, if I open 4 or more tabs in Chrome the computer will grind to a halt, and I gotta wait perhaps a few minutes for it to swap everything out then I can close the tabs. 150MB for a tab, that’s just way way way too much.

Personally I strongly dislike “virtual memory” in the sense of swapping to disk. I’d much rather get a (non-fatal) “out of memory” error than have the computer grind to a halt, which is what happens when a virtual memory computer goes a bit over its RAM. I don’t want to click a different window that I haven’t used for a while and have to wait for 3 minutes while the computer loads it from disk again and tries to figure out what to swap out. If we didn’t use swap, programmers (looking at you, Google) would be more careful not to waste memory.

Computers are not all that much more functional than they were in to 1990s, or even the 1980s, for regular office tasks such as wordprocessing and spreadsheets – and those computers although technically slower were actually more responsive in many cases because they did NOT grind to a halt due to swapping.

</rant>

How to Divide by Zero: a/0 = b ⟺ a = 0

TLDR: Division by zero is not as scary as it’s made out to be:

a/0 = b ⟺ a = 0

---

Division is multiplication, backwards. These two equations are exactly equivalent, by definition:

a/c = b

a = b×c

It’s easy to understand division by zero if we look at the equivalent multiplication.

a/0 = b

a = b×0

For any real number b:

a = b×0 = 0

a = 0

There are two cases with division by zero:

If a = 0, then a/0 = b is unconstrained, any real number b satisfies the equation. You can discard such an equation which does not constraint the result.

If a ≠ 0 then a/0 = b is contradictory. There is no real number b which satisfies that equation. This is still useful to know; “there is no answer” can be a sort of meta-answer. For example if trying to solve a system of equations of static forces, “there is no answer” might mean you need to consider a different design for your bridge!

There is no need to consider advanced concepts such as limits in order to fully understand division.

In short, a/0 = b is true if and only if a = 0.

If you see such an equation a/0 = b, you may simplify it to a = 0.

a/0 = b ⟺ a = b×0 ⟺ a = 0

a/0 = b ⟺ a = 0

I posted this here about a year ago: http://matheducators.stackexchange.com/a/5667/3287

error handling in C

I started writing a set of error handler macros for C, based on “Zed’s Awesome Debug Macros”

The implementation is quite ugly, and depends on a couple of GNU extensions.  This is not ideal, and I would like to improve it if possible.

https://github.com/sswam/kisskit/blob/master/kiss.h

The idea is to call functions via “wrapper macros”, which take care of checking for errors.  Here’s an example:

#include <stdlib.h>
#include <stdio.h>
#include <errno.h>
#include <string.h>
#include "kiss.h"

int main(void)
{
	char *space = NULL;
	FILE *fp = NULL;
	INFO("Hello");
	space = MALLOC(1000);
	space = REALLOC(space, 10000);
	fp = FOPEN("asdfasdf", "r");
	FCLOSE(fp);
	FREE(space);
	INFO("Bye");
	exit(0);
error:
	exit(1);
}

Output when I run it:

[INFO]  test1.c:11 main: Hello
[ERROR] test1.c:14 main: fopen failed: asdfasdf: No such file or directory

Output when I build it with -DNDEBUG and run it:

Hello
fopen failed: asdfasdf: No such file or directory

In most cases, I can add a wrapper macro with one short line of code:

#define MALLOC(...) CHK(malloc, __VA_ARGS__)
#define REALLOC(...) CHK(realloc, __VA_ARGS__)
#define FREE(ptr) (free(ptr), ptr = NULL)

#define FOPEN(path, mode) ({ char *p = path; FILE *rv = fopen(p, mode); CHECK(rv, "fopen failed: %s", p); rv; })
#define FCLOSE(fp) (ZERO(fclose, fp), fp = NULL)

The FOPEN macro is longer. I added the file name to the error message.

FREE is not checking for errors. It sets the pointer to NULL after the free. FCLOSE checks for errors and sets the FILE * to NULL.

I also wrote wrappers for some SDL and SDL_image functions, just a few so far.

Here is a short SDL example using the macros:

https://github.com/sswam/kisskit/blob/master/eg/test.c

and the equivalent code without the macros, which is much longer:

https://github.com/sswam/kisskit/blob/master/eg/test3.c

I’m in two minds about this. On the one hand, I think these wrappers can make C code shorter, safer, more readable, and more maintainable. On the other hand, it might be best to avoid macros, the implementation is ugly, and it uses two GNU extensions: ##__VA_ARGS__, and ({ statement expressions }).

Please let me know what you think. (Without being too scathing, please… I respond better to kindness!)

ramp-io

I wrote a simple program ramp-io, based on the redshift code, to read and write the xrandr gamma ramps for Linux / X11.  This enables me to define my own gamma ramps, and switch ramps quickly from the command line.  My preferred ramp is red-inv, dim inverse video with a low colour temperature (more red, less blue), and I set the LCD hardware brightness to maximum to reduce LED PWM flicker.  I find this is relatively easy on the eyes for work, compared to the normal glaring white backgrounds.

printf “%q ” in bash, to escape arguments

I learned a useful trick with the bash shell today.

We can use printf “%q ” to escape arguments to pass to the shell.

This can be useful in combination with ssh, in case you want to pass arguments containing shell special characters or spaces. It can also be used with su -c, and sh -c.

The following will run a command exactly on a remote server:

sshc() {
        remote=$1 ; shift
        ssh "$remote" "`printf "%q " "$@"`"
}

Example:

sshc user@server touch "a test file" "another file"

job control in the shell

Job control is a basic feature of popular UNIX and Linux shells, such as “bash”.
It can be very useful, so I thought I’d make a little tutorial on it…

^C    press Ctrl-C to interrupt a running job (you know this one!)
^\    press Ctrl-\ (backslash) to QUIT a running job (stronger)
^Z    press Ctrl-Z to STOP a running job, it can be resumed later
jobs  type jobs for a list of stopped jobs (and background jobs)
fg    type fg to continue a job in the foreground
bg    type bg to continue a job in the background
kill  kill a job, e.g. kill %1, or kill -KILL %2
wait  wait for all background jobs to finish

You can also use fg and bg with a job number, if you have several jobs in the list.

You can start a job in the background: put an &-symbol at the end of the command. This works well for jobs that write to a file, but not for interactive jobs. Things might get messy if you have a background job that writes to the terminal.

If you forget the % with kill, it will try to kill by process-id instead of job number.  You don’t want to accidentally kill PID 1!

An example:

vi /etc/apache2/vhosts.d/ids.conf
^Z
jobs
find / >find.out &
jobs
fg 2
^Z
jobs
bg 2
jobs
kill %2
fg

Fight Back against universal surveillance (aka Big Brother)

Today, FEBRUARY 11TH 2014, is “the day we fight back” against universal surveillance by our governments, and specifically the NSA (aka Big Brother).  Please add your voice to this protest, and add their black banner to your site/s.

https://thedaywefightback.org/

P.S. It seems that wordpress.com would charge me $30 to change the background colour of my page.  This is absurd, I will have to look for or likely make a better blog system.

svn ignore dammit

“Thank-you” svn for such a hideous, bogus and difficult way to ignore files.

So I wrote a script svn-ignore-all:

#!/bin/sh

# svn-ignore-all

# usage: 
#   1. run svn status to see what is going on at each step 
#   2. add or commit all files that you DO want to have in svn
#   3. remove any random files that you don't want to svn:ignore
#   4. run this script to svn:ignore everything marked '?' in output of `svn status`

svn status |
grep '^?' |
sed 's/^? *//' |
while read f; do
    d=`dirname "$f"`
    b=`basename "$f"`
    ignore=`svn propget svn:ignore "$d"`
    if [ -n "$ignore" ]; then
        ignore="$ignore
"
    fi
    ignore="$ignore$b"
    svn propset svn:ignore "$ignore" "$d"
done

Also, to ignore specific list of files / pathnames, we can use this variant svn-ignore. I guess svn-ignore-all should really be like xargs svn-ignore.

#!/bin/sh

# svn-ignore

# usage:
#   svn-ignore file/to/ignore ...

for f; do
    d=`dirname "$f"`
    b=`basename "$f"`
    ignore=`svn propget svn:ignore "$d"`
    if [ -n "$ignore" ]; then
        ignore="$ignore
"
    fi
    ignore="$ignore$b"
    svn propset svn:ignore "$ignore" "$d"
done

One more thing: I tend to pollute my svn checkouts with many random files. When it’s time to commit, I move those files into an ‘old’ subdirectory, and tell svn to ignore ‘old’.

Also posted here:  http://stackoverflow.com/questions/116074/how-to-ignore-a-directory-with-svn/19240497#19240497