What's New in v0.8.8

A release of fixes and build speed, found while writing FluxGX (a node-based modelling tool on sokol and Dear ImGui) against 0.8.7. Modules are now real namespaces.

Some checks are stricter. Read Stricter checks if a program that built with 0.8.7 no longer does. One rule changed, and the same program can compute something else with it: see Behavior change.

Faster builds

The object cache

The C and C++ sources of modules (and your own @cfile / @cppfile sources) are compiled to an object file once, and the cached object is linked afterwards. Only your own code is compiled on the following builds.

A Dear ImGui + sokol program0.8.70.8.8
First build6.9 s2.7 s (module sources compile in parallel)
Every build after it6.9 s0.8 s
  • Nothing to set up. It covers every module, modules added later and project sources.
  • An entry is reused only while the source and every header it includes are unchanged, for the same compiler and the same flags.
  • Used with clang and gcc on all three backends. The bundled TCC is not cached.
  • gx doctor shows its size; gx doctor --clear-obj-cache empties it; --no-obj-cache or GX_OBJ_CACHE=0 bypasses it; GX_OBJ_CACHE_DIR moves it.

See Getting Started.

Prebuilt module libraries

gx doctor --rebuild <module> builds any module that ships C / C++ sources into a static library: sokol, cimgui, cimgui_sokol, cimgui_raylib, json, xml, enet, clay, threading, and your own modules under modules/. It was raylib only. The recipe is read from the module’s own @cfile / @cppfile / @cflags lines, so there is nothing to configure, and gx doctor --rebuild all covers them all.

gx doctor --rebuild cimgui --use

With --use, a program that imports the module links the library and compiles none of its sources. The build goes back to the sources, with a note, when the library is missing (each GX version has its own) or older than the module’s sources; --no-prebuilt ignores it for one build.

The library is always the optimized build, also inside a -g build of your program. On build time alone it gives what a warm object cache gives.

Language and library

Modules are namespaces

import a.one as uno
import b.two

fn main() {
    print("{uno.value()} {b.two.value()}\n")    // both modules define value()
    var n: uno.Node = uno.Node{1}               // and both a struct Node
    var m: b.two.Node = { 2, 3 }
}

A function, global, const or type belongs to the module that defines it, so two modules may use the same name. Before, every name of every module shared one scope, and the second definition was an error (or, for a struct, quietly ignored).

  • A qualifier selects its module: util.f(), the full dotted path a.one.f() (which used to fail in the C compiler) or an alias uno.f(). It is checked: uno.f() with f in another module is an error.
  • Types are qualified the same way, wherever a type is written: n: uno.Node, array<uno.Node>, (*uno.Node)p, uno.Node{1}, sizeof(uno.Node).
  • A bare name is the one of the file’s own module, otherwise the one of the only import that defines it. When two imports define it, the error names both and the qualifiers to write.
  • Everything a module defines is public. pub is accepted and warns that it has no effect.
  • In the generated C a name keeps its spelling; only a name that two modules define becomes <module>__<name>.

The editor completes after uno. and goes to the definition through a qualified name. See The Language Tour.

Integer literals up to u64

const BASIS: u64 = 14695981039346656037    // above the i64 range
var h: u32 = (k ^ 2166136261) * 16777619   // with k: u32, all of it is u32
print("{BASIS} {BASIS >> 60}\n")           // 14695981039346656037 12

A literal is the smallest of i32, i64 and u64 its value fits, and beside an integer operand it takes that operand’s type where the value fits. A u64 prints as unsigned and folds as unsigned in constants.

sort() in n log n

array<T>.sort() and slice.sort() were an insertion sort: 255 000 numbers took 8 seconds. They are now an introsort (well under a second), and array<f32> / array<f64> sort on every backend.

T[] without a size is a slice

var g_args: str[] (no size, no initializer) is a slice, a pointer and a length, as main’s args: str[] is: g_args = args keeps the command line for later.

Containers as function parameters

fn total:i32(a: array<i32>) {             // read-only
    var t = 0
    for (i = 0:a.len() - 1) { t += a[i] }
    return t
}

fn squares(out a: array<i32>, n: i32) {   // the caller's array
    a.clear()
    for (i = 1:n) { a.push(i * i) }
}

var a: array<i32>
squares(&a, 4)
print("{total(a)}\n")                      // 30

array<T> and map<K,V> can be parameters. Before, a container had to be wrapped in a struct or passed as a slice. The rule is the one pointers already follow: without out the function can only read the container, with out it works on the caller’s and the call site says so with &. Nothing is copied either way. A container field is passed the same way (squares(&mesh.ids, 4)), and a channel parameter is always out (out c: chan<T>), because a channel is shared, never copied.

Not part of this: a function still cannot return a container, and a function-pointer type cannot name a container parameter.

dot on every vector type

var a = dot(vec2{3.0, 4.0}, vec2{1.0, 0.0})     // 3
var b = dot(v3a, v3b)
var c = dot(v4a, v4b)

dot takes two vectors of the same type: vec2, vec3 or vec4. It was vec3 only.

A constant expression as an array size

const ROWS = 4
const COLS = 4

struct Grid { cells: i32[ROWS * COLS] }
var edges: f32[COLS + 1]

The size of a fixed array may be an expression of integer literals and top-level integer consts (+ - * / % << >>, unary minus, parentheses). It was a number or one const name.

char and bytes convert to each other

fn is_digit:bool(c: char) { return c >= '0' && c <= '9' }

var c: char = s[0]          // indexing a str gives a u8
if (is_digit(s[i])) { }
var b: u8 = 'A'

A char and a one-byte integer (u8, i8, c_char, c_uchar) convert to each other without a cast, in declarations, assignments, arguments and return. A wider integer still needs (char)n.

out in a function-pointer type

struct NodeType {
    cook: fn(out *Ctx):bool
}

A function-pointer type may mark a pointer parameter out, in a field and in a parameter. It documents that the callee writes through the pointer.

io.fs: listing a directory

var entries: array<FsEntry>
if (list_dir("recipes", &entries)) {
    for (i = 0:entries.len() - 1) {
        print("{fs_entry_name(&entries[i])}\n")
    }
}
entries.free()

list_dir fills an array with the files and subdirectories of a directory, sorted by name; an entry has a name and is_dir. There was no way to read a directory before.

sokol.gfx: the whole API

sokol.gfx binds all of sokol_gfx: 121 structs, 150 functions and every SG_* constant, generated from the header. It was a hand-written file with 15 functions. Buffers, shaders, pipelines, offscreen passes and image views are usable from GX.

Names that Windows defines as macros

near, far, small, min, max, interface and hyper are ordinary names for variables, fields and functions on every platform. On Windows they did not compile, because the generated C includes <windows.h>.

Stricter checks

Each reports at compile time what used to fail later, in the C compiler, or not at all.

  • dot and cross check their arguments. dot takes two vectors of the same type and cross two vec3. The arguments were not checked at all before.
  • An indexed write through a read-only pointer parameter is an error: dst[i] = v with dst: *u8 needs out dst: *u8, as *dst = v and dst.field = v already did. So does dst[i]++, and a write to an element of a container behind a read-only pointer (g.pts[i] = v with g: *Geo), where only g.pts.push(v) was caught.
  • A read-only parameter cannot be passed on to an out parameter, neither itself nor the address of a place inside it (bump(&p.hits) with p: *Counter). It compiled, with a warning from the C compiler at best, and wrote through the read-only pointer.
  • A pointer argument must match its parameter. Any pointer was accepted for any pointer parameter, so take(&s) with s: *S passed a **S and the program read a pointer as the struct. A different number of pointer levels, or a different struct, is now an error (“‘s’ is already a pointer: pass ‘s’, not ‘&s’”). *void still takes any pointer.
  • A fixed array is not a parameter. a: f32[3] was read as a: f32, and v: *vec2[3] built on no backend. Both are refused where they are declared, with the fix: write a: []f32, which an f32[3] converts to at the call.
  • The arguments of a qualified call are checked, as those of a bare call always were: util.twice("abc") went to the C compiler.
  • A module sees what it imports itself. A file could use a module that only another module imported. Add the import (the bundled raylib modules now import raylib.core for its vector types).

Behavior change

A literal takes the type of the integer beside it, on either side. 1 - n with n: u32 is now a u32 (it was an i32), so a negative result wraps, as n - 1 always did. Write (i32)n where the sign matters.

Bugs fixed

Wrong results at run time

BackendWasNow
llvm-clangA local array<T>, map<K,V> or chan<T> of a vector, matrix or struct (var pts: array<vec3>) used an element size of 8 bytes: elements overlapped and memory was corruptedThe element’s real size
llvm-clangA str passed to a cstr parameter of a GX function was passed as the whole {ptr, len} pair, shifting every argument after itThe data pointer is passed
llvm-clangcstr == null compared null with null: always trueThe pointer is compared
Ca.len() on array<T> was unsigned, so on an empty array for (i = 0:a.len() - 1) ran instead of doing nothingi32, as in GX
llvm-clangsizeof(Point[4]) was the size of one PointThe size of the array
CAn f32 literal beside an f32 (x * 0.1, x <= 0.1, x += 1.0) was written as a C double, so the operation was done in double precision: results differed from the LLVM backends in the last bits, and comparisons with a literal could differ outright (x <= 0.1 was false for var x: f32 = 0.1).f32 arithmetic, on every backend: the literal is written as a C float
C, default compiler on WindowsA call with several arguments, one of them a nested float expression (clamp_f(a[i] + s * (r - 0.02), lo, hi)), received that argument overwritten by another one: a fault of the bundled tcc 0.9.27Such an argument is passed through an identity function when compiling with tcc, which it handles correctly
C, default compiler on WindowsA call with an a && b or a || b argument and a struct passed by value (pivot(lo, hi, bottom && !part)) crashed the program: the same compilerThe same remedy
all"{x}" with a str x was x itself: for a view (s.sub(...)) not a new string and not NUL-terminatedAn interpolated string is always a new, terminated string
allA u64 in a formatted string printed its low 32 bits (C) or as signed (LLVM)Unsigned, all 64 bits
allA u64 constant above the i64 range folded as signed: BIG >> 60 gave 1844674407370955161212
native llvmA function passing its own out container on to a read-only container parameter crashed the compiler, now and thenBuilds

Accepted by GX, rejected by the C compiler

WasNow
A str cast to a pointer type ((cstr)s, (*u8)s) cast the {ptr, len} structThe string’s data pointer
Arguments of an extension-method call were not converted between str, cstr and charConverted like the arguments of an ordinary call
fabsf with the default compiler on Windows: TCC could not compile its own <math.h> inline (“unknown constraint ‘t’“)Builds
sizeof(f32[65]), an array of a built-in type written outThe size of the array; the count may be a constant expression
On llvm-clang, an integer that is not a literal pushed into an array of a narrower type (bytes.push(n > 1 ? 1 : 0)) or returned from a function with a narrower return typeConverted, as on the other backends
A local named like a C keyword GX does not reserve (var long = 3)Written long_ in the C
A string literal over two lines: clang refuses the line break in CWritten as \n
String literals in a cstr[N] initializer, or beside a cstr in ? :, stayed GX stringsC strings
A function with a read-only pointer parameter assigned to a function-pointer field: clang refused const T* against T*Cast to the field’s type
A global str[] assigned from argsA slice (see above)
On llvm-clang: str.sub() with an i32, a method of a global Arena, a global array<T> passed or copied, an f64 into a vec2 component, a slice indexed inside a vector operatorBuild
On native llvm: a field taken straight off a call’s result, f(x).fieldBuilds
a.one.f(), the full path of a dotted moduleCalls f (see Modules are namespaces)
al.Node{3} compiled to garbageAn initializer of the module’s Node

Warnings from clang

A program built with clang (-O1 and up, or --cc clang) no longer shows warnings that come from GX itself: static const const char* for a cstr constant, an f32 constant written as a double (so the expression was computed in double precision; it is now an f32, as on the LLVM backends), fopen and the like marked deprecated by MSVC’s C library, and a read-only pointer passed to an extern or stored in a variable. FluxGX’s editor went from 90 such warnings to none.

Rejected by GX, although valid

WasNow
var h = SOME_CONST made h constantAn ordinary variable
return "literal" from a function returning cstr, returning a cstr as str, return null as cstrreturn converts as arguments and assignments do
var c: cstr = null and var f: fn(i32):i32 = null, and assigning null to one later (h.text = null)Accepted
A function-pointer field of an extern struct could not return a pointerParsed like any other field
A vector field read through a read-only pointer could not be passed or returned by value: len(w.dir) with w: *Wire gave “expected vec2, got vec2”A value read from something read-only is an ordinary value
A module-level var g: array<T> on llvm-clangSupported, as on the other backends
.ptr of a slice or fixed array on the native llvm backend (fell back to the text emitter)Lowered natively

Tests

The compiler suite is 122 programs, each run on c, llvm-clang and native llvm, plus tests/obj_cache.sh and tests/prebuilt_lib.sh for the build features.