The Bluejay Operating System
Building Bluejay
Bluejay uses the home-grown Jmk
build system, which is basically just a GNU
m4 script that generates makefiles from Jmk
files — makefiles with some
custom macros.
To build a fresh clone of Bluejay the first thing you will need to do is run
bin/jmk
to generate your makefiles for you. You should get some output like
this:
Processing ./boot/initrd/Jmk
Processing ./src/kernel/dri/ata_pio/Jmk
Processing ./src/kernel/dri/ahci/Jmk
Processing ./src/kernel/dri/pci/Jmk
Processing ./src/kernel/Jmk
Processing ./src/mkinitrd/Jmk
Processing ./src/lisp/Jmk
Then just build using src/kernel/Makefile
. There are a few additional
targets for your convenience:
qemu
builds and launches the kernel using QEMU’s SeaBIOSqemu-iso
builds a GRUB ISO and launches using QEMUinstall
builds a GRUB ISO and installs it toboot/bluejay.iso
debug
launches kernel in QEMU and launches GDB in the terminal.debug-wait
launches kernel in QEMU and starts a GDB server onlocalhost:1234
. This is recommended if you want to debug since you can connect to it from vscode or any other IDE..vscode/launch.json
is set up to work with this so you can debug the kernel very easily.
In order to build Bluejay you will need the following dependencies:
gcc gcc-multilib nasm qemu-system-i386 make m4 python3 awk
There are some additional dependencies for building a GRUB ISO but I don’t remember them at the time of writing.
Common Issues
If when launching Bluejay in QEMU with make qemu
you see the message
[ DEBUG ] mb.mods_addr = <whatever>, 0x<whatever>
Followed by a page fault (#PF
), your QEMU is out of date. You can either run
the GRUB ISO (with make qemu-iso
), which is slower, or upgrade your QEMU to
at least version 6.0.
This is because prior to 6.0 QEMU’s integrated bootloader did not support multiboot modules. This message is caused by the kernel attempting to read a module that was loaded incorrectly.
JMK 2 Build System
JMK2 is a rewrite of the JMK build system. I am slowly porting Bluejay to JMK2 instead of the legacy M4-based JMK build system.
JMK2 is used to generate makefiles for each project in Bluejay. A
project is a directory with a Jmk2
file (case sensitive). Each
project produces a single output based on some sources.
The script bin/jmk2
looks in the source tree for Jmk2
files,
and process each one into the corresponding Makefile. It accepts
option definitions with the -D
flag, eg ./bin/jmk2
-DSOME_OPTION=123
. You can also specify the C compiler, assembler,
and linker to use with the -c
, -a
, and -l
flags,
respectively.
Here is an example Jmk2 file:
init hello # hello is the name of the project
srcs hello.c world.c # the source files this project uses
type executable # the preset type of project this is
Each line consists of a command (init
, srcs
, type
) and its
arguments. The commands are documented here:
- init name [target]
Initializes the project with a given name. The
name
is currently unused, but should be set to a descriptive identifier.target
is the name of the target that the project generates. By default it is the same asname
. For an executable, this could behello
orhello.exe
. For a shared library,libhello.so
, etc.
- preset preset_name
Applies the preset
preset_name
. A preset is a function defined in the::presets
namespace which makes some changes to the project state.These are the default presets:
freestanding
Changes thecflags
to build a freestanding binary (without linking the standard library).optimize
Changes thecflags
andasmflags
to enable compile-time optimizations.32
Tells the compilers to produce a 32 bit build.debug
Tells the compilers to enable debug information in the resulting builds (enables DWARF symbols).warn
Enables useful warnings and-Werror
.nasm
Setsnasm
as the default assembler.
- presets preset_a [preset_b]...
Applies all the given presets in order. Identical to calling
preset
once for each argument.
- cflag string
Adds
string
to the::cflags
variable, which will be passed to the C compiler.
- cflags string_a [string_b]...
Adds multiple strings to the
::cflags
variable, the same as callingcflags
repeatedly.
- asmflag, asmflags
Same as
cflag
,cflags
but for the::asmflags
variable.
- option name default_value
If the option
name
has not been specified when invokingbin/jmk2
, sets the value of the option todefault_value
. Options can be read with::options(option_name)
.
TODO: finish!
Architecture
This document seeks to provide a brief overview of Bluejay architecture. This should be a good starting point for understanding the code.
Bluejay is exclusively a multiboot kernel, it neither provides nor supports alternative bootloaders.
The bootloader (probably GRUB) will initially run the code in boot.s
. This
is where it all begins. This code sets up segmentation and paging and maps the
higher-half of virtual memory (everything above 0xC0000000
) to the kernel.
At first it only maps 8 megabytes, more memory can be mapped on request.
After moving to high memory the kernel jumps to C code and enters kmain
in
main.c
. This is the highest level procedure in the kernel, which sets up
kernel services and drivers one at a time.
This includes VGA, keyboard, and PCI drivers, as well as paging and preemptive multi tasking.
Multi tasking
Multi tasking is handled by code in task.c
. It is first initialized in
init_tasks
, which sets up the initial task. Once this is called kernel
threads can be spawned at will.
Every clock tick an interrupt is triggered (see clock.c
for timing) which
causes a task switch to occur. Bluejay uses a simple round-robin scheduler, and
there is no way for tasks to voluntarily give up their processing time (even in
the case of blocking IO operations). task.c
contains the implementation of
the scheduler.
Drivers
So far drivers must be written either using plain in
and out
instructions or on top of the existing PCI driver.
PCI Device Drivers
PCI device drivers must register a struct pci_device_driver
in order to
interface with a certain device (or class of devices). See
include/kernel/dri/pci/pci.h
for details.
A PCI device driver must pass an instance of this structure to
pci_register_device_driver
(in include/kernel/dri/pci/pci.h
. If
supports
returns true, (for example, if the class and subclass of the
struct pci_device
are supported by teh driver) init
will be called. At
this point the driver may do whatever it wishes with the PCI device, although
all blocking operations should be done in another thread (using spawn_thread
in include/kernel/task.h
for example).
Bluejay Filesystem
Filesystem drivers are still a work in progress. To test a file system you will
want to create and mount a virtual block device. The makefile in src/kernel
will generate an hd0_ext2.img
EXT2 disk image for you automatically. The
default size is 32 megabytes, but you can create your own of any size if you
want. Once the image has been created it will be loaded by QEMU automatically.
In order to write to the virtual hard disk from your host operating system you
should mount it. The make mount
command in src/kernel
mount the image to
$(BLUEJAY_ROOT)/mnt
. If you are using an EXT2 filesystem you should probably
change the owner of that directory once it is mounted so that you can write to
it.
Virtual Filesystem
The Bluejay VFS is heavily inspired by UNIX. It relies on inodes and a tree of
file nodes. The source can be found in src/kernel/vfs.c
. This also exports a
very low-level API for dealing with files – including the usual read(),
write(), readdir(), etc – but this should not be used for much longer. A high
level API utilizing file descriptors will be implemented to make this simpler.
Filesystem Drivers
The current filesystem driver(s) available in Bluejay are:
ext2
Read-only support, write support is in progress
Creating a Virtual Drive in QEMU
By default make qemu
will load hd0_$(FS).img
as the virtual hard drive
for Bluejay. FS
defaults to ext2
but can be set in your Jmk.options
to any value. If this file does not exist it will be created using
mkfs.$(FS)
, ie mkfs.ext2
by default. The default size of the file system
is 35 megabytes, although you can create one of any size manually if you want.
35 megabytes is plenty for testing though.
The make mount
command will mount the current virtual hard drive in
$(ROOT)/mnt
(where $(ROOT)
is the root directory of the Bluejay sources,
not /
). This command requires superuser privileges. If you want to give your
(host) user account write permissions use chown -R user:group /path/to/mnt
where user
and group
are the user and group you want to own the files.
Currently Bluejay ignores file permissions so it doesn’t matter who you set the owner to.
Lisp Standard Library
This provides documentation for every built-in function in the Lisp standard library. It is not auto-generated, please update this documentation if you change the API in any way.
In general every user-facing API in the standard library should be documented here.
(x ...)
represents a listx
.& body
means that the rest of the list is represented bybody
.[something]
means thatsomething
is optional.
Top-level primitives
These are “functions” that can only appear at the top-level of the program. This means they can’t be nested in any other expressions.
- (defun function-name (args ...) & body)
Defines a function
function-name
that takesargs
and evaluatesbody
.function-name
is quoted, not evaluated.(defun say-hi (name) (print "Hi, ") (print name)) (say-hi "Joe") ; "Hi," ; "Joe"
- (defmacro macro-name (args ...) & body)
defmacro
is to macros asdefun
is to functions. Whenmacro-name
is called, whatever it evaluates to will be compiled.Note that internally this compiles a function the same way all other functions are compiled, meaning you can call any lisp function from a macro definition and it will work as expected.
(defun double (n) (+ n n)) (defmacro call-with-4 (whatever) (print "this was run at **compile time**") (print whatever) ;; ``whatever`` expands to the form passed to this macro, in this case ;; ``double``. (list whatever 4)) (print (call-with-4 double)) ; "this was run at **compile time**" ; 'double ; 8
Functions
- (if condition true-condition [false-condition])
Evaluates
condition
, if it is truthy (non-nil
)true-condition
is evaluated. Otherwisefalse-condition
is evaluated. Iffalse-condition
is not provided andcondition
isnil
,if
will evaluate tonil
.(print (if (= 2 3) "2 = 3" "2 /= 3")) ; 2 /= 3
- (let1 (variable binding) & body)
Evaluates
binding
and binds it tovariable
, then evaluatesbody
. Afterbody
is evaluatedvariable
is unbound.(let1 (greeting (greet "John")) (do-something greeting) (print greeting)) ; greeting is no longer bound
- (gc)
Force the garbage collector (GC) to run.
- (car pair)
Return the first item in
pair
.(car (cons 'a 'b)) ;=> 'a
- (cdr pair)
Return the second (last) item in
pair
.(cdr (cons 'a 'b)) ;=> 'b
- (cons a b)
Return a cons-pair containing
a
andb
.
- (print val)
Print out
val
to standard output. This will not be formatted as an s-expression, but in a manner more similar to the internal representation.
- (list & items)
Returns a cons-list of items.
(list 1 2 3) ; is the same as (cons 1 (cons 2 (cons 3 nil)))
- (quote form)
Returns form without evaluating it.
'(cons a b) ; or (quote cons a b) ; is the same as (list 'cons 'a 'b)
- (lambda (args ...) & body)
Creates an anonymous function (closure). This function uses lexical scope meaning that any free variables (variables bound outside this lambda definition) are “captured” by the closure. You can call this function with
funcall
(to be implemented) orapply
.(let1 (number 3) (let1 (adds-number-to (lambda (n) (+ n number))) (print (apply adds-number-to '(5))))) ; 8
- (apply function (args ...))
Call
function
withargs
and return the result. Note that since this is a Lisp-2 (i.e. functions and variables do not share the same namespace) you need to pass a function object (i.e. a lambda or quoted function).
Kernel Logging
Drivers and other kernel components may write log messages to the default output
(currently only VGA since it is the only display target implemented) using
kprintf
in include/kernel/log.h
. Additional defines in the same file may
help differentiate different types of log messages (i.e. errors, debug
information, etc).
kprintf(OKAY "Something succeeded\n");
kprintf(ERROR "Something failed :(\n");
// etc, see log.h for details
Bluejay is an operating system inspired by UNIX and early Lisp machines. Currently it only targets x86. There are no plans to port to other platforms.
This documentation should provide an introduction to compiling, developing, and using Bluejay.