mem_alloc.h File Reference

#include "system/macros.h"
#include <glib.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <malloc.h>

Include dependency graph for mem_alloc.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Macros
#define	DT_IS_ALIGNED(x)   __builtin_assume_aligned(x, DT_CACHELINE_BYTES)
	Promise the compiler that x is already cacheline-aligned, and return it.
#define	DT_CACHELINE_BYTES   64
#define	DT_CACHELINE_FLOATS   16
#define	DT_CACHELINE_PIXELS   4
#define	DT_ALIGNED_ARRAY   __attribute__((aligned(DT_CACHELINE_BYTES)))
	Align an object on a cacheline boundary, so AVX2 can load it whole.
#define	DT_ALIGNED_PIXEL   __attribute__((aligned(16)))
	Align a 4-float pixel on 16 bytes, enough for SSE. Same struct-member caveat as DT_ALIGNED_ARRAY, with a weaker requirement that malloc() happens to meet on most platforms – which is exactly why misuse survives testing here and not there.
#define	dt_free(ptr)
	g_free() ptr and set it to NULL, skipping both if it is already NULL.
#define	dt_free_align(ptr)
	Release memory from dt_alloc_align() and set ptr to NULL.

Functions
static gboolean	dt_is_aligned (const void *pointer, size_t byte_count)
	Is pointer aligned on a byte_count boundary? Pure test, no side effect.
static size_t	dt_round_size (const size_t size, const size_t alignment)
	Round size UP to the next multiple of alignment.
static size_t	dt_round_size_sse (const size_t size)
	Round size up to the next multiple of 64.
static void *	dt_alloc_align_internal (size_t size)
	Platform back-end for dt_alloc_align(). Call dt_alloc_align() instead.
void *	dt_alloc_align (size_t size)
	Allocate cacheline-aligned memory.
static void	dt_free_gpointer (gpointer ptr)
	g_free() one pointer, with the signature GDestroyNotify wants.
static void	dt_free_align_ptr (void *mem)
	Platform back-end for dt_free_align(). Call dt_free_align() instead: this one takes the pointer by value and so cannot NULL the caller's variable.
static void *	dt_calloc_align (size_t size)
	dt_alloc_align() followed by a zero fill.
static float *	dt_alloc_align_float (size_t pixels)
	Allocate pixels floats, cacheline-aligned and marked as such.
static float *	dt_calloc_align_float (size_t pixels)
	dt_alloc_align_float() followed by a zero fill.
static void *	dt_check_sse_aligned (void *pointer)
	Runtime-checked counterpart to DT_IS_ALIGNED().
static void	memset_zero (void *const buffer, size_t size)
	Set the memory buffer to zero as a pack of unsigned char.

Macro Definition Documentation

DT_ALIGNED_ARRAY

#define DT_ALIGNED_ARRAY   __attribute__((aligned(DT_CACHELINE_BYTES)))

Align an object on a cacheline boundary, so AVX2 can load it whole.

Warning

On a STRUCT MEMBER this attribute constrains the member's offset within the struct – it cannot constrain where the struct itself lands. A struct carrying one of these must therefore be allocated by dt_alloc_align() (or live on the stack, where the compiler honours it); plain malloc()/g_malloc() guarantee far less alignment, and the member then straddles a cacheline. The result is a crash or silently wrong pixels in vectorised code, at the point of USE, with nothing wrong at the point of allocation.

Definition at line 80 of file mem_alloc.h.

DT_ALIGNED_PIXEL

#define DT_ALIGNED_PIXEL   __attribute__((aligned(16)))

Align a 4-float pixel on 16 bytes, enough for SSE. Same struct-member caveat as DT_ALIGNED_ARRAY, with a weaker requirement that malloc() happens to meet on most platforms – which is exactly why misuse survives testing here and not there.

Definition at line 85 of file mem_alloc.h.

DT_CACHELINE_BYTES

#define DT_CACHELINE_BYTES   64

Definition at line 66 of file mem_alloc.h.

DT_CACHELINE_FLOATS

#define DT_CACHELINE_FLOATS   16

Definition at line 67 of file mem_alloc.h.

DT_CACHELINE_PIXELS

#define DT_CACHELINE_PIXELS   4

Definition at line 68 of file mem_alloc.h.

dt_free

#define dt_free

(

ptr

)

Value:

  if(!IS_NULL_PTR(ptr))        \
  {                            \
    g_free((void *)(ptr));     \
    *(void **)(&(ptr)) = NULL; \
  }

g_free() ptr and set it to NULL, skipping both if it is already NULL.

Parameters

ptr	an lvalue holding a pointer. It is assigned NULL, so a temporary, a cast expression or a function result does not compile – deliberately: the whole point is that no caller is left holding a dangling copy.

Warning

Pairs with g_malloc()/g_strdup() and the rest of the GLib family, NOT with dt_alloc_align(). Use dt_free_align() for that.

Expands to a bare if with no do { } while(0) wrapper, so if(c) dt_free(p); else ... fails to compile with "else without a previous if". Brace the branch. It is a compile error rather than a silent misbinding, so it cannot reach runtime – but it does force braces where none should be needed.

Definition at line 171 of file mem_alloc.h.

dt_free_align

#define dt_free_align

(

ptr

)

Value:

  if(!IS_NULL_PTR(ptr))               \
  {                                   \
    dt_free_align_ptr((void *)(ptr)); \
    *(void **)(&(ptr)) = NULL;        \
  }

Release memory from dt_alloc_align() and set ptr to NULL.

Parameters

ptr	an lvalue, for the same reason as dt_free(). NULL is accepted and ignored.

Warning

Must be used for – and only for – dt_alloc_align(), dt_calloc_align(), dt_alloc_align_float() and dt_calloc_align_float(). See dt_alloc_align() for what crossing the two families costs on Windows.

Same bare-if expansion as dt_free(): brace the branch in an if/else.

Definition at line 214 of file mem_alloc.h.

DT_IS_ALIGNED

#define DT_IS_ALIGNED

(

x

)

__builtin_assume_aligned(x, DT_CACHELINE_BYTES)

Promise the compiler that x is already cacheline-aligned, and return it.

Warning

This ASSERTS, it does not align: the compiler is free to emit aligned loads and stores on the strength of the promise. Applying it to a pointer that is not in fact aligned is undefined behaviour, and typically shows up as a SIGSEGV in vectorised code far from the allocation. Only use it on memory from dt_alloc_align() and friends, or on a stack object declared DT_ALIGNED_ARRAY. Use dt_check_sse_aligned() when the alignment is not known statically.

Definition at line 51 of file mem_alloc.h.

Function Documentation

dt_alloc_align()

void * dt_alloc_align

(

size_t

size

)

Allocate cacheline-aligned memory.

The single entry point for buffers that vectorised code will touch. The returned block is aligned on DT_CACHELINE_BYTES and its usable size is size rounded UP to that boundary, so writing the rounded size is safe and reading it is defined.

Parameters

size	requested byte count.

Returns

the block, or NULL when the allocation fails – which happens in practice on the pixel path, where buffers are hundreds of megabytes. Callers must check.

Warning

Release ONLY with dt_free_align(). On Windows this comes from _aligned_malloc() and passing it to free()/g_free()/dt_free() corrupts the heap; on POSIX the two paths happen to coincide, which is precisely why such a mismatch passes every Linux test and fails on Windows only.

See also

dt_calloc_align() for the zeroed variant, dt_alloc_align_float() for float counts.

Definition at line 507 of file darktable.c.

References dt_alloc_align_internal(), and size.

Referenced by _brush_get_pts_border(), _camera_all_ids(), _dt_colorchecker_patch_init(), _group_get_mask(), _import_copy_file(), _lens_ids_for_camera(), _load_jpg(), _masks_draw_creation_session_forms(), _orient_scope_buf(), _polygon_crop_to_roi(), _polygon_init_ctrl_points(), _refresh_preview_module_histogram_for_hash(), _set_test_path(), dt_alloc_align_float(), dt_cache_get_with_caller(), dt_cache_seed(), dt_cairo_sharpen_surface_rgb24(), dt_calloc_align(), dt_calloc_align_float(), dt_colorchecker_patch_array_init(), dt_colorspaces_add_profile(), dt_colorspaces_enumerate_profiles(), dt_focuspeaking(), dt_folder_survey_destination_preview(), dt_image_cache_allocate(), dt_iop_levels_compute_levels_automatic(), dt_iop_tonecurve_draw(), dt_ioppr_set_pipe_input_profile_info(), dt_masks_border_find_self_intersections(), dt_masks_iop_combo_populate(), dt_masks_write_masks_history_item(), dt_mipmap_cache_alloc(), dt_mipmap_cache_allocate_dynamic(), dt_noiseprofile_get_matching(), extract_color_checker(), reload_defaults(), and update_preview_cb().

dt_alloc_align_float()

static float * dt_alloc_align_float ( size_t  pixels )

inlinestatic

Allocate pixels floats, cacheline-aligned and marked as such.

Returns

the block, or NULL on failure. Release with dt_free_align().

Warning

pixels is a COUNT OF FLOATS, not a byte count, and the multiplication is not checked for overflow – validate suspicious dimensions before calling.

Definition at line 235 of file mem_alloc.h.

References dt_alloc_align(), and DT_CACHELINE_BYTES.

Referenced by _allocate_curves(), _brush_get_gravity_center(), _brush_get_points_border(), _build_clut(), _build_clut(), _draw_graph_background(), _extract_patches(), _group_get_mask(), _polygon_get_gravity_center(), build_gui_kernel(), build_pixel_kernel(), commit_params(), dt_iop_tonecurve_draw(), dt_ioppr_init_profile_info(), extract_color_checker(), init_pipe(), process(), process_cl(), pseudo_solve(), solve_hermitian(), and validate_color_checker().

dt_alloc_align_internal()

static void * dt_alloc_align_internal ( size_t  size )

inlinestatic

Platform back-end for dt_alloc_align(). Call dt_alloc_align() instead.

Returns

cacheline-aligned memory of at least size bytes (rounded up), or NULL if the allocation failed. Must be released with dt_free_align().

Definition at line 122 of file mem_alloc.h.

References DT_CACHELINE_BYTES, dt_round_size(), and size.

Referenced by dt_alloc_align().

dt_calloc_align()

static void * dt_calloc_align ( size_t  size )

inlinestatic

dt_alloc_align() followed by a zero fill.

Returns

the zeroed block, or NULL on failure. Release with dt_free_align().

Note

Only the size bytes asked for are zeroed, not the padding dt_alloc_align() rounds the block up to.

Definition at line 225 of file mem_alloc.h.

References dt_alloc_align(), and size.

Referenced by _ensure_sample_cache_capacity(), _group_get_mask(), default_init_pipe(), dt_colorspaces_prepare_conversion(), dt_iop_init_pipe(), dt_masks_copy_used_forms_for_module(), find_patch(), gui_init(), gui_init(), init_pipe(), init_pipe(), init_pipe(), and parse_cht().

dt_calloc_align_float()

static float * dt_calloc_align_float ( size_t  pixels )

inlinestatic

dt_alloc_align_float() followed by a zero fill.

Returns

the zeroed block, or NULL on failure. Release with dt_free_align().

Definition at line 241 of file mem_alloc.h.

References dt_alloc_align(), and DT_CACHELINE_BYTES.

Referenced by _chromaticity_gradient_stage_cl_selftest(), and dt_masks_debug_rasterise().

dt_check_sse_aligned()

static void * dt_check_sse_aligned ( void *  pointer )

inlinestatic

Runtime-checked counterpart to DT_IS_ALIGNED().

Returns

pointer marked aligned for the compiler when it really is cacheline-aligned, and NULL when it is not – the return value is the test result, so ignoring it and using pointer anyway defeats the purpose. Does not allocate; ownership is unchanged.

Definition at line 251 of file mem_alloc.h.

References DT_CACHELINE_BYTES, and dt_is_aligned().

dt_free_align_ptr()

static void dt_free_align_ptr ( void *  mem )

inlinestatic

Platform back-end for dt_free_align(). Call dt_free_align() instead: this one takes the pointer by value and so cannot NULL the caller's variable.

Definition at line 193 of file mem_alloc.h.

Referenced by _free_box_gpointer().

dt_free_gpointer()

static void dt_free_gpointer ( gpointer  ptr )

inlinestatic

g_free() one pointer, with the signature GDestroyNotify wants.

Parameters

ptr	taken BY VALUE, so unlike the dt_free() macro this cannot NULL the caller's variable – the assignment in the body only clears the local copy and is dead. Use it where a GLib container needs a free function, not as a general-purpose free.

Definition at line 184 of file mem_alloc.h.

Referenced by __attribute__(), _apply_preferences(), _ask_and_delete(), _chain_clear(), _choose_gpx_callback(), _colorspaces_destroy(), _datetime_undo_data_free(), _dev_auto_apply_presets(), _dnd_folder_import_job_cleanup(), _draw_paths(), _dt_style_update_iop_order(), _event_dnd_get(), _exif_xmp_read_data(), _exif_xmp_read_data_export(), _expand_geometry_and_misc(), _expand_image_and_version(), _expand_rating_and_metadata(), _film_import1(), _folder_survey_job_run(), _geotag_undo_data_free(), _gradient_slider_destroy(), _history_undo_data_free(), _history_write_state_release(), _hm_backup_cleanup(), _hm_build_id_set_from_mod_list(), _hm_build_last_history_by_id(), _hm_build_last_history_by_id_from_history(), _hm_build_next_map_from_ids(), _hm_build_override_map(), _hm_build_prev_map_from_ids(), _hm_cached_order_applicable(), _hm_report_build_moved_set(), _hm_report_update_arrows(), _hm_restore_dest_from_backup(), _hm_topo_build_constraint_ids(), _hm_topo_build_id_info_table(), _hm_topo_merge_cleanup(), _hm_topo_resolve_incompatible_constraints(), _ioporder_get_current_order_name(), _liquify_warp_points(), _metadata_view_update_values(), _pop_undo(), _print_button_clicked(), _refresh_display_track(), _resync_xmp_duplicates(), _save_last_tag_used(), _set_printer(), _setup_selected_images_list(), _styles_init_source_dev(), _styles_prepare_source_dev(), _thumbtable_dnd_scan_folders_and_import(), _undo_metadata_free(), _update(), _upgrade_data_schema_step(), _upgrade_library_schema_step(), _view_map_draw_other_locations(), _write_xmp_id(), append_styles(), apply_clicked(), build_global_distortion_map(), camera_menu_fill(), cleanup(), clear_search(), delete_clicked(), dt_accels_window(), dt_bauhaus_combobox_from_conf(), dt_bauhaus_widget_set_label(), dt_cleanup(), dt_conf_init(), dt_control_crawler_show_image_list(), dt_control_delete_images_job_run(), dt_control_export_job_run(), dt_control_import_data_free(), dt_dev_cleanup(), dt_dev_history_undo_start_record_locked(), dt_dev_pixelpipe_cache_init(), dt_dev_pixelpipe_cleanup_nodes(), dt_dev_snapshot_capture(), dt_dng_opcode_process_opcode_list_2(), dt_exif_read_blob(), dt_exif_xmp_read(), dt_film_remove_empty(), dt_folder_survey_absorb_new_files(), dt_folder_survey_count_new_files(), dt_folder_survey_init(), dt_get_selected_files(), dt_gui_preferences_enum(), dt_gui_set_themes(), dt_history_get_items_as_string(), dt_hm_batch_state_cleanup(), dt_image_cache_deallocate(), dt_image_read_duplicates(), dt_import_cleanup(), dt_init(), dt_ioppr_change_iop_order(), dt_ioppr_deserialize_iop_order_list(), dt_ioppr_deserialize_text_iop_order_list(), dt_ioppr_get_iop_order_list(), dt_ioppr_rebuild_iop_order_from_modules(), dt_ioppr_set_default_iop_order(), dt_ioppr_update_for_modules(), dt_lib_export_metadata_configuration_dialog(), dt_lib_export_metadata_set_conf(), dt_masks_free_form(), dt_metadata_clear(), dt_metadata_set(), dt_metadata_set_import(), dt_metadata_set_list(), dt_style_repository_normalize_multi_priority(), dt_styles_create_from_image(), dt_styles_get_item_list_as_string(), dt_styles_save_to_file(), dt_styles_style_data_free(), dt_tag_import(), dt_view_manager_cleanup(), dtgtk_gradient_slider_multivalue_clear_stops(), export_clicked(), finalize_store(), free_chart(), free_labels_list(), free_location(), free_params(), geometry_record(), geometry_record(), get_query_string(), gui_cleanup(), gui_cleanup(), gui_reset(), import_clicked(), import_preset(), init_presets(), lens_menu_fill(), list_match_string(), load_watermarks(), main(), main(), main(), modify_roi_in(), parse_cht(), refresh_watermarks(), run_style_scenarios(), set_params(), store(), update_profile_list(), and write_image().

dt_is_aligned()

static gboolean dt_is_aligned ( const void *  pointer, size_t  byte_count  )

inlinestatic

Is pointer aligned on a byte_count boundary? Pure test, no side effect.

Definition at line 88 of file mem_alloc.h.

Referenced by dt_check_sse_aligned().

dt_round_size()

static size_t dt_round_size ( const size_t  size, const size_t  alignment  )

inlinestatic

Round size UP to the next multiple of alignment.

Parameters

size	byte count to round. Zero rounds to zero.
alignment	must be non-zero; zero divides by zero.

Returns

the rounded size, always >= size.

Definition at line 99 of file mem_alloc.h.

References size.

Referenced by _prepare_resampling_plan(), dt_alloc_align_internal(), and dt_round_size_sse().

dt_round_size_sse()

static size_t dt_round_size_sse ( const size_t  size )

inlinestatic

Round size up to the next multiple of 64.

Note

Hardcoded 64, NOT DT_CACHELINE_BYTES: on Apple Silicon a cacheline is 128, so this rounds to half a cacheline there. It is named for the SSE register width it was written for; do not substitute it for dt_round_size(size, DT_CACHELINE_BYTES).

Definition at line 111 of file mem_alloc.h.

References dt_round_size(), and size.

Referenced by _dt_masks_dynbuf_growto(), fast_eigf_surface_blur(), and fast_surface_blur().

memset_zero()

static void memset_zero ( void *const  buffer, size_t  size  )

inlinestatic

Set the memory buffer to zero as a pack of unsigned char.

Parameters

buffer	destination, at least size bytes. Not NULL-checked.
size	number of bytes to clear. The body writes one unsigned char at a time, so any size is handled; the "multiple of 8" this once required is no longer a constraint.

Note

The intent is a zero fill a compiler may not elide, as memset_s() guarantees in C11. A plain byte loop is not that guarantee – an optimiser is allowed to recognise it and call memset() anyway – so do not rely on this to scrub secrets.

Definition at line 270 of file mem_alloc.h.

References k, and size.

Referenced by lmmse_demosaic(), process(), and xtrans_markesteijn_interpolate().