Data in Production

Caching Nixos build outpts

Any Nixos package can be built, using a nix function src which takes its source code and outputs Nix-store path of the built package, and it can be used from that location. Nix is a pure language and this output is reproducible and this data is immutable. This data can be cached this outputs can be cached. While Caching data in a database typically involves storing frequently accessed data to improve retrieval speed and reduce load on the backend, while caching in Cachix for Nix involves storing pre-built binaries to avoid rebuilding packages, thus speeding up package management and deployment across the machines of homelab, in cloud, datacenter (with terraform) etc. Terraform service can be built with the following Nix expression

inherit (inputs.terranix.lib) terranixConfiguration;
terraform = pkgs.terraform;
      infra = import ./terraform/default.nix { inherit system pkgs inputs; };
      tfConfig = terranix.lib.terranixConfiguration {
        inherit system;
        modules = [ infra ];
      };
      cachix-deploy-lib = cachix-deploy-flake.lib pkgs;
            terraformConfigurations.default = terranixConfiguration {
        extraArgs = { inherit terranix system pkgs; };
        #   {
        #     inherit system
        #       #inherit (inputs.)
        #       pkgs; # <-- Here we inherit terranix, system and pkgs so it's available in our configuration.nix file as a function parameter
        #   };
        modules = [ ./terraform/default.nix ];
      };

# default.nix
            { pkgs, system, ... }: {
  # Example Terraform config via Terranix
  provider.aws.region = "us-east-1";

  resource.aws_s3_bucket.my_bucket = {
    bucket = "my-nix-generated-bucket";
    acl = "private";
    # acl obsolete use sue  aws s3 bucket
    # set api check skip or credntials
    tags = {
      Name = "my-nix-bucket";
      Environment = "production";
    };
  };
}

Pros ¹

Native, reproducible. Integrates with NixOS modules directly and Shares host’s Nix store automatically.

Lazy eval thanks to partial application allows reusable code which can be referred in other modules possible by partial attributes {attr1, attr2 ? type:{}} (where type is certain type of value eg bool or set or string) and lib.mkif and optional attrs & recursive attrsets in defining a key,value and then declaring key1, key2, key3 etc, with neset attrs. Nix function for terrafornm can be written like this.

Nix writers and bash Strings

Regular bash shell interprets a string litrally ( try echoing [:6-*,’s81 ) , if that seems like a cherry picked example, try to echo any out pout from the script below output you get from the following

import random
import string

def generate_string(length):
    if length < 8 or length > 64:
        raise ValueError("Length must be between 8 and 64 characters.")

    allowed_chars = string.ascii_letters + string.digits + "!?-_.:,;()=[]\"'?'+%*#~@{}<>$^?/\\"
    return ''.join(random.choice(allowed_chars) for _ in range(length))

# Example usage
result = generate_string(10)
print(result)

Okay? If still feel tricked, my freind you RTFM! I don’t.

So any bash script in nix writer like pkgs.writeShellScript etc ll need the script to be quoted as ‘#script ..’; so that you can put special chars for nix attrs like $ in it and Nix can parse it , wihtout side effects. An example of not doing it , in absence of any special chars can be

# just imperative terminal commands, no env setup with export and accessing them later with special chars like $
services.redis.servers.nextcloud = {
    enable = true;
    bind = "::1";
    port = 6379;
  };

  systemd.services.nextcloud-setup.serviceConfig.ExecStartPost = pkgs.writeScript "nextcloud-redis.sh" ''
    #!${pkgs.runtimeShell}
    nextcloud-occ config:system:set redis 'host' --value '::1' --type string
    nextcloud-occ config:system:set redis 'port' --value 6379 --type integer
    nextcloud-occ config:system:set memcache.local --value '\OC\Memcache\Redis' --type string
    nextcloud-occ config:system:set memcache.locking --value '\OC\Memcache\Redis' --type string
  '';

OTOH different nix builders like any other anoymous lambda (takes no other parameters), that you write as nix module, requires differnt number of args, and without sufficinet number of args provided, it won’t build.

To know the arguments of the modules you ve written, you can use

nix eval .#nixosConfigurations.thinkpadTest-vm.config.disko

or

nix eval .#nixosConfigurations.thinkpadTest-vm.options.disko

and applying | jq . ² or attrNames to these outputs

which gives [ “checkScripts” “devices” “enableConfig” “imageBuilder” “memSize” “rootDisk” “rootMountPoint” “testMode” “tests” ]

Nix is dynamically typed so there re no type checks as you write, you to evel in nix repl as you write , which is too much work. That said , you don’t need a browser to see avaialble options, you can just start with the skeleton of what you need in a flake and you can get all the options with Pressing tab in the nix eval lines above, or to get them one by one nix develop -j 50 .#nixosConfigurations.thinkpad.config.services.gitlab-runner.services and it expands to runner for docker executor gitlab-pm-docker and for shell executor gitlab-pm-shell. fwiw , they don’t come out of box .

Using linux services for storing and hosting it

This can be done with containerised linux or services using Nextcloud, Minio. As in, if minio s3 is listenting to 9001 to act as distribut web store for nextcloud listening to 8080, which uses pgsql listens to eg /run/postgresql.sock for storage and redis listens to /run/redis-nextcloud/redis.sock for disributed caching , locking of data.

Many NixOS service modules create Unix sockets, and the service owner (root or a specific user) determines the permissions and access control for that socket file in the filesystem, the services here like minio,nextcloud .. and neither of them re owned by nixos home,system user , despite their files being in /var/lib. The good news is that these services default to tcp ports and we need not reconfigure them to create unix sockets. Now there re nixos services bound to listne to a port , to use them on the domain of the service which listens to another port say portx i ll need another reverse proxy from port to portx

virtualHosts =
     let
       base = locations: {
         inherit locations;

         forceSSL = true;
         enableACME = true;
       };
       proxy = port:
         base { "/".proxyPass = "http://127.0.0.1:" + toString (port) + "/"; };
     in
     {
       # this service doesn't have a hostname option hence manual
       # minio doesn't need a vhost to a web ui
       "${config.services.nextcloud.config.objectstore.s3.hostname}" = proxy 9000
         // {
         listen = [{
           addr = "127.0.0.1";
           port = 8080;
         }];
         # VERY IMPORTANT for S3v4 signatures
         extraConfig = ''
           proxy_set_header Host $host;
           proxy_set_header X-Forwarded-Host $host;
           proxy_set_header X-Forwarded-Proto $scheme;
           proxy_set_header Connection "";
           proxy_http_version 1.1;
         '';

         root = "/var/lib/minio/data/nextcloud";
         locations."/var/lib/nextcloud/data/" = {
           # done above by overriding
           #proxyPass = "http://127.0.0.1:9005";
         };
       };

Deploying and automating these services with gitlab runner

Gitlab ci/cd scripts use docker images of linux to run these services.

Self hosting, Network

Reverse proxy and ingress.

It is not same as NAT port forward. try tailscale serve 3000 on terminal and it ll tell you to use the flag to tun off https on 443, as that’s where its already serving and hence it gets the ssl certs for the host. To self host tailscale and listen to another port , you need a reverse proxy with nginx or caddy with a virtualhost. Caddy saves you from having to configure ssl certs as it autoconfigures it. Not to mention its plugins for tailscale and dns.

Make your HTTP (or HTTPS) network service available using a protocol-aware configuration mechanism, that understands web concepts like URIs, hostnames, paths, and more. The Ingress concept lets you map traffic to different backends based on rules you define, you do them maually with a reverse proxy and declarativelt wuth ingress for clustred servers , manage by kubernetes api or otherwise. In general its helpful as a traffice controller when the infra has max load, to pass any futher traffic, just like a real traffic signal A reverse proxy for localhost typically listens on a TCP port, allowing external clients to connect through that port. In contrast, a reverse proxy using a Unix socket communicates through a file system path, which can be more efficient for local inter-process communication and avoids port conflicts sought outcome.

Firewalls

My node , peers use wireguard tunnel wth tailscale and a general iptable firewall, to allow acctepting ports like 5432 for pgsql or forward my online port virtual nic as internal interfaces for my nixos-VM , is just some command in the firewall besides allowing tailscale0 and wg0 TCP ports.

networking.firewall.extraCommands =  ''
        iptables -A INPUT -p tcp --dport 5432 -j ACCEPT
        iptables -A FORWARD -i vnet0 -o wlp0s20f3 -j ACCEPT
        iptables -A FORWARD -i wlp0s20f3 -o vnet0 -m state --state RELATED,ESTABLISHED -j ACCEPT
        iptables -t nat -A POSTROUTING -s 192.168.100.0/24 -o wlp0s20f3 -j MASQUERADE
'';

An example of which I use in my flake ³

Options need not be browsed online, just start with skeleton and per TAB age

Cons

Not ideal for persistent state (unless you configure shared mounts) , & Less control over snapshotting or systemd integration. I didn’t do it with docker, as that islolates gpu passthrough and I needed an accelerated VM. Although the config includes a gitlab-runner to deploy it over gitlab, withe executors docker,shell or kubernets.

storage Nextcloud allows to configure object storages like OpenStack Swift or Amazon Simple Storage Service (S3) or any compatible S3-implementation (e.g. Minio or Ceph Object Gateway) as primary storage replacing the default storage of files. ⁴

Difference

The difference between br0 in Ir original and vnet1 as used in the recommended approach comes down to their roles and what they represent in Ir network setup:

br0 (Linux Bridge Interface)

In Ir original config, br0 is bridged directly to Ir wireless interface wlp0s20f3. This attempts to make the VM appear as if it’s directly on the host’s local network. Bridging Wi-Fi interfaces is often problematic because mywireless drivers and consumer-grade routers do not support being enslaved to Linux bridges or do not handle promiscuous mode and broadcast traffic well. br0 represents a shared Layer 2 segment where connected devices see each other directly.

vnet1 (Virtual NIC Interface)

Concurrent NAT and Bridge Networking

I may have multiple virtual network configurations (NAT and bridged) coexisting, but different VMs or NICs use different network interfaces, but cannot use NAT hostfwd-style port forwarding on bridged mode interfaces. VM’s network configuration (IP addresses, gateways, DNS) must correspond to the network mode: NATed network IPs (like 10.0.2.x) for user-mode or NAT network, LAN IPs for bridged mode networks.

Firewall and Port Forwarding Firewall (iptables or nftables) rules on the host affect NAT and forwarding for virtual networks. Port forwarding works with the NAT network but is not applicable in bridged mode. I need extra firewall and NAT rules on the host bridging interface (br0) if I want NAT or isolation or selective access.

Segmentaion Policy I tag the incoming connections by their purpose, eg shopping, banking, work, communication etc,to prioritize which is them loads quicker.When I emulate this with this vm, I also emulate my relevant hardware acceleartors, gpu drivers, kernel modules and display service.

As a general limitations these VM’s are not visible to extenal devices even on the same lan be it ethernet or wifi lan. As a workaround I use an overlay network with tailscale ⁵.

share build critical state data and secrets from host, using virtiofs with bindmounts, which re chrooted jails.

Use Case

Tailscale access to multiple VMs or containers behind a single gateway VM, or when I need to segment a network within my VM environment.

In QEMU, network devices can be emulated, allowing guest operating systems to interact with virtual network interfaces. The register_netdev() function plays a role in ensuring that these virtual devices are properly registered within the Linux kernel running in the guest environment.

This function is used in the QEMU rocker switch driver to allocate and register a network device for each physical switch port, allowing for control traffic and higher-level constructs like bridges and VLANs.

_xhcg instruction in Linux boot processes is used to exchange values between registers or between a register and memory, which can be important for managing data during system initialization. It helps in tasks like setting up the stack or managing memory addresses efficiently.

Raw code insertion

• i ffi/hsbracket.c

  anubis ⁶

Footnotes

# A helper function for agenix secrets
 agenixSecrets = { userHome ? "/home/xameer", files, }: {
        age = {
          identityPaths = [
            "/etc/ssh/ssh_host_ed25519_key"
            "/etc/ssh/ssh_host_rsa_key"
            "${userHome}/.config/sops/age/keys.txt"
            "${userHome}/.ssh/age_id_rsa"
            "${userHome}/.ssh/age-id_ed25519"
          ];

          secrets = files;
        };
      };
 # resued in a module
#only for systemd services
        (agenixSecrets {
          files = {
            # nextcloud secrets
            "nextcloud-admin-password" = {
              file = ./secrets/nextcloud-admin-password.age;
              owner = "nextcloud";
              group = "nextcloud";
            };
            };
          # more file paths in other modules
          })
          #which can be referenced in any other module
              secretKey = config.age.secrets.minio-secret-key.path;
              #or literal string not a nix $ of rec
              export MINIO_ACCESS_KEY=$(cat /run/agenix/minio-access-key)
          # or can be declaterd in a let block of a module and later accessed there in the in block.

Copyright (C) 2013-2025 Xameer . All rights reserved.