Posts

Each Gated DeltaNet head in Qwen3.5-0.8B carries a $128\times128$ state matrix. I wanted to know how much of that state the trained model depends on. I measured its singular values, then repeatedly removed lower-energy modes during inference. On the tested web-text documents, keeping only 48 modes per head left next-token loss essentially unchanged. Keeping 12 increased loss by just 0.005 nats. Several retrieval tasks also tolerated substantial truncation. But the result depended on the task: at 64K, truncation improved variable tracking and sharply hurt common-word counting....

The basic struct of string consists of three members: struct string { char* mPtr; // dynamically allocated memory size_t mSize; // the length of the string size_t mCapacity; // the size of allocated memory }; Allocating memory for small strings (e.g., empty string with a null \0 character) is wasteful. Hence, to avoid this waste, most implementations of string structs apply Small String Optimization (SSO), which stores small strings directly within the string object on the stack, rather than allocating memory dynamically on the heap....

In a prior post, I talked about list initialization, which is differs from initializer_list discussed here. Though personally, I don’t find initializer_list really useful as I never used it in my projects. list initialization is a general syntax using {} for initializing a variety of variables and objects. initializer_list is a template class representing a lightweight, read-only array of elements, typically used in constructors or functions. initializer_list promotes safety, flexibility and modern tone compared to the raw array....

constexpr is a keyword in C++ that allows the compiler to evaluate expressions at compile time. This is a powerful feature that can significantly optimize performance by reducing runtime overhead. However, I mainly use it for type-related operations. I seldom apply it to data-related tasks, since defining data with constexpr requires constant values, which is rarely feasible in my projects. Code with constexpr #include <stddef.h> #include <string_view> #include <algorithm> #include <cstdio> template<size_t N> class FixedString { size_t mSize{}; char mData[N]{}; public: FixedString() = default; // Constructor that computes string length at compile time constexpr FixedString(const char* str) : mSize{std::char_traits<char>::length(str)} { std::copy_n(str, size(), mData); } constexpr size_t size() const { return mSize; } constexpr std::string_view string_view() const { return {mData, mSize}; } }; template<size_t N> constexpr auto make_fixed_string(const char (&str)[N]) { return FixedString<N>{str}; } constexpr const static FixedString<50> x{"Hello, embedded World!...

In C++, the term “stateless” typically refers to a type (class or struct) that: Has no non-static data members, meaning it does not store any instance-specific information. Does not maintain any internal state or data that varies between objects of that type. Stateless types are often empty classes used for utility purposes, such as: Custom deleters for smart pointers. #include <memory> #include <iostream> struct EmptyDeleter { void operator()(int* ptr) const { delete ptr; std::cout << "Deleted\n"; } }; int main() { std::unique_ptr<int, EmptyDeleter> ptr(new int(42)); std::cout << "Size of unique_ptr: " << sizeof(ptr) << " bytes\n"; // 8 bytes return 0; } Tags for template metaprogramming....

[[no_unique_address]] since C++20 [[no_unique_address]] applies to user-defined types (e.g., empty or stateless classes or structs). It does not apply to fundamental types (int, float, etc.), as they always require memory for storage. The attribute optimizes memory layout by allowing empty or stateless user-defined types to overlap memory locations, improving efficiency without violating the C++ object model. Motivation Prior to C++20, Empty Base Optimization (EBO) allowed an empty base class to take zero space when it was inherited by another class....

In a struct, the padded bytes depend on the alignment requirement of the next member following the current member, because the compiler must ensure proper and efficient access to memory. Alignment Requirement: Each data type has a required alignment, which is typically a power of two. For example: char: 1-byte alignment int: 4-byte alignment long (on a 64-bit system): 8-byte alignment double: 8-byte alignment Padding: When laying out struct members, if the next member needs stricter (i....

Definition of an Empty Class An empty class is a class that: Contains no non-static data members. May include: Member functions (including operator() or constructors), but these do not contribute to the class size. Static data members, because these are shared across all instances and are not part of the object layout. Does not use virtual functions or polymorphism, which would require the inclusion of a vtable pointer. Inherits from another empty class, as the derived class can still remain empty due to Empty Base Optimization (EBO)....

Recently, I watched this talk by Andreas Fertig at CppCon'24. He discussed some very interesting topics, including new C++ features and union-based optimizations for efficient C++ programming. To fully understand this talk byte by byte, I tried to re-implement the examples and experiment on my own, figuring out details with the help of ChatGPT. But I quickly found myself going down a rabbit hole of unfamiliar concepts that I’m not quite up to grasp yet....

A performance evaluation is conducted on an Nvidia L40, comparing the 100-iteration access times of device vectors with half, half2, and float types. Each vector was initialized with 1024*1024 elements, but for the half2 type, two elements were packed into a single vector entry. Hence, two randomness are tested for half2 type: random access per half2 and random access per half. Access Type Data Type Vector Size Allocated Memory Time (ms) Random half 1M 2MB 4....