Datagram builder as a C extension#259

Open

methodmissing wants to merge 21 commits intomainfrom

datagram-builder-c-extension

Commits on Mar 30, 2020

Initial C extension implementation

csfrancis
authored and
methodmissing
committed
Enable warnings + pedantic flag and allow for easier debugging by setting ENV var STATSD_EXT_DEBUG
methodmissing
committed
Implement DatagramBuilder#generate_generic_datagram as a C function with a static buffer
methodmissing
committed
Introduce support for conditionally enabling GC.stress for the test suite with env var GC_STRESS
methodmissing
committed
Prevent buffer overflows in DatagramBuilder#generate_generic_datagram, favoring broken packets to undefined behavior
methodmissing
committed
Introduce a bounded cache for DatagramBuilder#normalize_tags used from the C ext only (normalize_tags is a pure function)
methodmissing
committed
Let the C ext be a prepended module as opposed to redefinition which also allows the normalized tags cache to be decoupled from Ruby land
methodmissing
committed
Let the normalized tags cache be a compile time option
methodmissing
committed
No need to memset the buffer to 0s for 4096 bytes on every call as we are very careful with offsets, even when preventing overflows
methodmissing
committed
Also extract a bounded normalized metrics name cache (perhaps normalized cache limits should be configurable at runtime, idk?)
methodmissing
committed
The gist of this change is moving normalized caches to symbol tables instead of hashes. Along with that came several requirements which also illustrates how to implement wrapped structs and plug th…
methodmissing
committed
Extract normalize_name_fast_path which is guaranteed to not do a function call, still has O(n) complexity but cheaper than str hash + symbol table lookup
methodmissing
committed
Fix warnings about losing integer precision
methodmissing
committed
Introduce a fast path for sample rate Float and Fixnum types that prevents a Ruby String allocation per datagram
methodmissing
committed
Avoid Array alloc + concat for when both default and given tags are specified
methodmissing
committed
Hide #generate_generic_datagram and #normalize_name from Ruby and direct dispatch to them from the respective metric helper functions
methodmissing
committed
Prefer a strhash table for the normalized names cache
methodmissing
committed
Move the global string allocations to builder struct members instead
methodmissing
committed
Perform the empty default tags predicate check on init
methodmissing
committed
Let append_normalized_tags prefer a bool return too
methodmissing
committed

Commits on Apr 1, 2020

Explicitly set freed struct members and the builder to NULL in datagram_builder_free
methodmissing
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Datagram builder as a C extension#259

Datagram builder as a C extension#259
methodmissing wants to merge 21 commits intomainfrom
datagram-builder-c-extension

Commits on Mar 30, 2020

Initial C extension implementation

Enable warnings + pedantic flag and allow for easier debugging by setting ENV var STATSD_EXT_DEBUG

Implement DatagramBuilder#generate_generic_datagram as a C function with a static buffer

Introduce support for conditionally enabling GC.stress for the test suite with env var GC_STRESS

Prevent buffer overflows in DatagramBuilder#generate_generic_datagram, favoring broken packets to undefined behavior

Introduce a bounded cache for DatagramBuilder#normalize_tags used from the C ext only (normalize_tags is a pure function)

Let the C ext be a prepended module as opposed to redefinition which also allows the normalized tags cache to be decoupled from Ruby land

Let the normalized tags cache be a compile time option

No need to memset the buffer to 0s for 4096 bytes on every call as we are very careful with offsets, even when preventing overflows

Also extract a bounded normalized metrics name cache (perhaps normalized cache limits should be configurable at runtime, idk?)

The gist of this change is moving normalized caches to symbol tables instead of hashes. Along with that came several requirements which also illustrates how to implement wrapped structs and plug th…

Extract normalize_name_fast_path which is guaranteed to not do a function call, still has O(n) complexity but cheaper than str hash + symbol table lookup

Fix warnings about losing integer precision

Introduce a fast path for sample rate Float and Fixnum types that prevents a Ruby String allocation per datagram

Avoid Array alloc + concat for when both default and given tags are specified

Hide #generate_generic_datagram and #normalize_name from Ruby and direct dispatch to them from the respective metric helper functions

Prefer a strhash table for the normalized names cache

Move the global string allocations to builder struct members instead

Perform the empty default tags predicate check on init

Let append_normalized_tags prefer a bool return too

Commits on Apr 1, 2020

Explicitly set freed struct members and the builder to NULL in datagram_builder_free