directory / file duality

UNIX neophytes are often to be heard raving about how “everything is a file”. In UNIX, things are simple, after all, and this is to be celebrated, we are told.

What they really mean to say is that “every process accesses data via file descriptors using the open(), close(), read(), and write() system calls”, but neither does that roll off the tongue as easily nor sound like a reason for celebration. Now, this uniformity is all well and good from an API perspective, but if everything were indeed a file, then every file, directory, and network socket would be addressable by name system-wide (or perhaps in the same namespace if one is thinking in terms of Linux containers), and not just per-process file descriptor), and the celebrations would be back on.

This article is not about that, though.

In the current POSIX setup, directories are opened via opendir(), read by readdir(), and closed by closedir(). These three are library functions, not system calls (and perhaps this is the problem). Regular files are opened via open(), read from by read(), written to by write(), and closed by close().

But imagine if a given path on a system could be opened either as a file or as a directory depending on the desired usage. What user-level data representations would this enable? What would this prevent? What, if anything, would this added functionality break?

This would allow a normal-looking regular file such as some/where/file.dat to possess “sub-resources” (this is not an official term, just one I’ll use in this article, and they’ll appear as underlined italic for clarity) such as some/where/file.dat/index-001.txt, some/where/file.dat/summary.txt, or some/where/file.dat/results.txt. An entire directory tree could exist in the same places as these sub-resources given as examples.

Similarly, a directory such as some/where/amazing.app/ which looks like an application directory could be opened as a file and therefrom one could some metadata might be readable, such as a set of compatibility requirements, or application signatures.

In short then, under this scheme, all files may be opened as directories, and all directories may be opened as files. The former gives the ability to add multiple files of metadata to an existing directory of data, for example. The latter allows for applying metadata labels to a directory, among other possibilities.

references https://www.microsoft.com/en-us/research/uploads/prod/2011/10/WhatIsAFile.pdf

HP Z240 SFF / M.2 SSD

M.2 SSD is Samsung MZV7S2T0BW This is a 2TB device.

PCI carrier board is HP MS-4365 Make sure you get one with a heatsink and two thermal conductive pads. One of the pads is slightly thicker than the other. The thicker one goes between the heatsink and the M.2 SSD. The thinner one probably won’t be any use.

With the heatsink, the M.2 SSD will run at about 45C. Without the heatsink, it will run at about 65C or higher. It will throttle its performance at these higher temperatures, and presumably wear out sooner rather than later.

HP Z240 SFF has four PCI slots. Install the MS-4365 carrier board into the 1 PCI Express Gen3 slot x 16 mechanical / x 4 electrical (LP, half length) slot with the low-profile tang. Connect the board to the terminal on the motherboard for the activity light. Note that this is not a SATA device, so the activity light on the front of the exterior will not illuminate when this device is err, active.

references

HP Z240 Small Form Factor Workstation Specifications

Samsung 970 EVO Plus 2 TB PCIe NVMe M.2 (2280) Internal Solid State Drive (SSD) (MZ-V7S2T0)

Quotes / random

“The key to understanding complicated things is to know what not to look at, and what not to compute, and what not to think.” Gerald Jay Sussman

“Everything comes to us that belongs to us if we create the capacity to receive it.” Rabindranath Tagore

“You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.” Buckminster Fuller

“A huge vocabulary is not always an advantage. Simple language… can be more effective than complex language, which can lead to stiltedness or suggest dishonesty.” John Gardner, The Art of Fiction

the next display server

security Security matters are not considered here. If this concerns or bothers you, please stop reading.

display server A display server is something which provides the service of display to something else. That “something else” is a client program. The terminology is sometimes confusing because the user interacts with the client program via the display server, the latter being a program running on the user’s device. This setup is very useful when one needs to have a graphical experience with a program running on a remote system.

remark However, the main effect is to confuse new users with the ambiguous terminology. This is the only situation of which I know where the user controls a client thing through a server thing.

browser The web browser is a case of arrested development. It’s not a display server, but it’s where web applications display themselves. Much has been written about “the browser as operating system”. This sounds clever, but does not really provide much insight, and is only true for a narrow sense of the term “operating system”.

browser apps these occupy an anonymous ground running in the browser on the desktop or on a mobile device.

electron apps these live in a peculiar location — running in a browser which can only execute JavaScript — but one that is used as a desktop application.

universality Given that digital content is made up of many formats, we need a tool to be able to (at least) render all of them — to literally browse collections of local and remote content, and to do so without having to install another application on one’s device.

renderer A universal document renderer is able to gain new abilities to display newly-encountered media types (such as audio), and newly-encountered encoding formats (such as MP3, OGG, JPEG, MOV, …), and does so without interrupting the user, except perhaps for confirming the addition of the new ability.

content types A universal document renderer has a bootstrap architecture, and broadly speaking, is little more than a framework for rendering plugins of various media types. This demotes the DOM / HTML / CSS combination from its current triumvirate status to be that of “just another content type”, and permits new expressions of interactive media.

transfer protocols HTTP could also lose its place as the dominant transfer protocol. URL notation is sufficient to enable the multi-protocol handling central to the bootstrapping nature of the system. HTTP remains as a basic transport for “DOM / HTML / CSS” content and renderer implementations.

implementations Code for rendering lives in repositories on the network. Universal document renderers are directed by content type metadata to fetch implementations of decoders for newly-encountered content types from these repositories. Once fetched, they remain cached locally by the framework until a new version is available, or they are no longer required and may be deleted. Universal document renderers may also check repositories periodically to ensure they have the desired (usually the latest) versions.

conclusion Browsers are lacking in flexibility to be classified as universal document renderers. Browsers have not gained the capability of rendering all possible document formats. In practice, the dominant document format is DOM via HTML and CSS, with PDF closely behind. Video (and to a lesser extent audio), have their own codecs but this is decided mainly by the main video sites and browser producers.

However, as browsers have evolved, they have effectively become a possible successor to display servers such as X11. This may be attributed to the efforts of developers of web frameworks, more than to those of browser authors.

The “next display server” is a universal document renderer, incremental in nature, and invisible in action. A display server is a system-level concept. A universal document renderer is a user- and application-level concept.

Strongtalk-2020 / beginnings

Strongtalk-2020 is the name I have given to of one of my recreational computing projects. It’s the continuation of the work done in the late 1990s by an all-star cast of programmers who went on to create many other amazing things including, but by no means limited to:

  • the Hotspot JIT compiler for the Java VM
  • the V8 Javascript VM
  • the Dart programming language
  • the Newspeak programming platform
  • the Java programming language specification
  • warehouse-scale computing at Google

This is quite an array of achievements — and was more than enough to pique my interest in the Strongtalk platform and investigate what work would be required to bring it up to date to run on a modern-day computer system.

Performing such a task is (at least) an interesting thought experiment, and there have been a number of attempts by people far more capable than me to resuscitate it.

Here are some of the challenges facts in the project visible from the outset:

  • 32-bit computing model
  • C++ 98 dialect of C++, and little use of the C++ standard library.
  • diabolical use of the preprocessor
  • Windows as the primary platform
  • Project appears to have been abandoned several times

Undeterred by the above, I started to see if I could get it compiled on a modern Linux installation. I consider Debian 10 to be the best option for what I need in 2020. Your experiences and requirements may be different. 🙂

I consider myself fortunate to have been involved in an ambitious porting job of an ANSI C99 project to many different UNIX platforms, as well as Win32 and MacOS X. (I was mainly involved in the UNIX part of things.) Although I made a few decisions then I would not make now, the experience has stuck with me to ensure that whatever one creates on a computer is usable by others. Emboldened by this experience I felt sure that I could manage it, and even if the project never made it to a worthwhile checkpoint, the task would be educational.

It’s always good to start these sort of things at the end and work backwards to the current status to see what’s required. These were the rough targets I had in mind.

  • Use the latest revision of the C++ standard, C++20, it having just been agreed by the C++ working group in February 2020.
  • Convert the code to be 64-bit only and drop support for 32-bit architectures.
  • Run on both Linux and Windows.
  • Conservatively make use of modern-and-popular C++ libraries, for important-but-peripheral aspects of the system such as logging and unit tests.
  • Slightly less conservatively, make use of modern and well-tested libraries, for important-and-central aspects of the system which required updating. The only component to which this applies is the x86-64 runtime macro assembler for the JIT compiler.
  • Even less conservatively, refresh the GUI toolkit.

Some of the less precisely defined goals are:

  • Keep the source tree the same shape where possible.
  • Improve the naming of classes and variables (usually this means making them longer and more descriptive)

the 2020 standard workstation

In the late 1990s, Professor Daniel J. Bernstein maintained on his web pages what he called “the standard workstation”, but as the early 2000s wore on, he no longer updated the specifications. This has stuck in my mind as a good concept.

Here is my equipment list for 2020. I don’t have all of it, but most of it is easily available. All of it just reflects my personal tastes and experience; you are not expected to agree with any of it 🙂

Keyboard Lenovo ThinkPad Compact USB Keyboard with TrackPoint
Mouse Integrated into the previous item. 🙂
Display Dell UP3017
Graphics NVIDIA GM107GL aka Quadro K1200
Disk Samsung 860 PRO SSD
CPU Intel i7 or Xeon @ 3.4 GHz
RAM As much as you can afford that will fit.

All in all, the best option for my use right now seems to me to be the HP Z240 SFF (small form-factor) workstation. Although they are no longer manufactured by HP, they are available to buy online at many places. Even better, HP are still producing BIOS updates for them.

HP Z240 SFF / sound

remarks

The system uses a Realtek ALC221 chipset, but with some HP-specific changes. To get it to behave properly, the following configuration must be applied.

files to add / edit

/etc/modprobe.d/snd_hda_intel.conf

# /etc/modprobe.d/snd_hda_intel.conf

options snd_hda_intel beep_mode=0
options snd_hda_intel enable_msi=1
options snd_hda_intel model=alc221-hp-mic
options snd_hda_intel patch=snd_hda_intel.fw
options snd_hda_intel power_save=0
options snd_hda_intel power_save_controller=N
options snd_hda_intel probe_only=0,1

/lib/firmware/snd_hda_intel.fw

# /lib/firmware/snd_hda_intel.fw

[codec]
0x10ec0221 0x103c802e 0

[pincfg] 
0x12 0x403c0000
0x14 0x01014020
0x17 0x411111f0
0x18 0x411111f0
0x19 0x03a19020
0x1a 0x02a11030
0x1b 0x0181303f
0x1d 0x40500001
0x1e 0x411111f0
0x21 0x0221102f

With the above in place:

  • the front microphone jack is active
  • both the front and rear headphone / line out jacks are active.

setting up

use alsamixer to disable “Auto-Mute Mode”