Welcome to kwutil’s documentation!

The kwutil Module

Read the docs	https://kwutil.readthedocs.io
Gitlab (main)	https://gitlab.kitware.com/computer-vision/kwutil
Github (mirror)	https://github.com/Kitware/kwutil
Pypi	https://pypi.org/project/kwutil

The Kitware utility module.

This module is for small, pure-python utility functions. Dependencies are allowed, but they must be small and highly standard packages (e.g. rich, psutil, ruamel.yaml).

• kwutil package
  • Submodules
    • kwutil.copy_manager module
      • CopyManager
        • CopyManager._operation_name
        • CopyManager._unsubmitted_report()
        • CopyManager._worker_func()
        • CopyManager.report()
        • CopyManager.submit()
    • kwutil.fsops_managers module
      • _FilesystemOperationManager
        • _FilesystemOperationManager._worker_func
        • _FilesystemOperationManager._operation_name
        • _FilesystemOperationManager.run()
      • _copy_worker()
      • _move_worker()
      • DeleteManager
        • DeleteManager._worker_func()
        • DeleteManager._operation_name
        • DeleteManager.submit_many()
        • DeleteManager.submit()
      • CopyManager
        • CopyManager._worker_func()
        • CopyManager._operation_name
        • CopyManager.submit()
        • CopyManager.report()
        • CopyManager._unsubmitted_report()
      • MoveManager
        • MoveManager._worker_func()
        • MoveManager._operation_name
        • MoveManager.submit()
        • MoveManager._check()
      • remove_empty_dirs()
      • _unsubmitted_info()
      • _pathinfo()
    • kwutil.partial_format module
      • partial_format()
      • _PartialFormatter
        • _PartialFormatter.vformat()
        • _PartialFormatter._vformat()
      • _test_partial_format()
      • subtemplate()
      • fsubtemplate()
      • _get_stack_frame()
    • kwutil.process_context module
      • ProcessContext
        • ProcessContext._infer_static_properties()
        • ProcessContext._infer_dynamic_properties()
        • ProcessContext.is_running
        • ProcessContext.is_started
        • ProcessContext.dump()
        • ProcessContext.__call__()
        • ProcessContext.write_invocation()
        • ProcessContext._timestamp()
        • ProcessContext._hostinfo()
        • ProcessContext._osinfo()
        • ProcessContext._pyinfo()
        • ProcessContext._meminfo()
        • ProcessContext._cpuinfo()
        • ProcessContext._gpuinfo()
        • ProcessContext._machine()
        • ProcessContext.start()
        • ProcessContext.flush()
        • ProcessContext.stop()
        • ProcessContext._start_emissions_tracker()
        • ProcessContext._flush_emissions_tracker()
        • ProcessContext._stop_emissions_tracker()
        • ProcessContext._device_info()
        • ProcessContext.add_device_info()
        • ProcessContext.add_disk_info()
      • jsonify_config()
      • Reconstruction
      • main()
    • kwutil.slugify_ext module
      • slugify()
      • smart_truncate()
    • kwutil.util_environ module
      • envflag()
    • kwutil.util_eval module
      • RestrictedSyntaxError
      • restricted_eval()
      • EvalException
      • ValidationException
      • CompilationException
        • CompilationException.exc
      • ExecutionException
        • ExecutionException.exc
      • SafeAST
        • SafeAST.allowed
        • SafeAST.generic_visit()
      • evalidate()
      • safeeval()
    • kwutil.util_exception module
      • add_exception_note()
    • kwutil.util_hardware module
      • get_cpu_mem_info()
      • get_cpu_info()
      • get_mem_info()
      • disk_info_of_path()
      • _device_is_hdd()
      • _zfs_status()
      • _torch_device_info()
      • _HardwareComponent
        • _HardwareComponent.summary()
      • CPUs
        • CPUs.summary()
        • CPUs._discover_v1()
      • Memory
        • Memory.summary()
        • Memory._discover_v1()
      • GPUs
        • GPUs.summary()
        • GPUs._discover_v1()
      • Motherboard
        • Motherboard.summary()
        • Motherboard._discover_v1()
        • Motherboard._discover_dmi_info_linux_non_admin()
      • Disks
        • Disks.summary()
        • Disks._discover_v1()
      • Networking
        • Networking.summary()
        • Networking._discover_v1()
      • Peripherals
        • Peripherals._discover_v1()
      • _hardware_nested_system_info()
      • _tup_to_nested()
      • Hardware
        • Hardware.report()
        • Hardware.cpus()
        • Hardware.memory()
        • Hardware.gpus()
        • Hardware.disks()
        • Hardware.motherboard()
        • Hardware.networking()
        • Hardware.peripherals()
    • kwutil.util_json module
      • debug_json_unserializable()
      • ensure_json_serializable()
      • find_json_unserializable()
      • Json
        • Json._load_filepointer()
        • Json.load()
        • Json.loads()
        • Json.dump()
        • Json.dumps()
        • Json.coerce()
        • Json.find_unserializable()
        • Json.ensure_serializable()
        • Json.debug_unserializable()
    • kwutil.util_locks module
      • Superlock
        • Superlock.THREAD_LOCKS
        • Superlock.GLOBAL_THREAD_KEY
        • Superlock.GLOBAL_APPNAME
        • Superlock.GLOBAL_LOCK_FNAME
        • Superlock._debug()
        • Superlock.global_lock_fpath
        • Superlock.acquire()
        • Superlock.release()
    • kwutil.util_parallel module
      • BlockingJobQueue
        • BlockingJobQueue.has_room()
        • BlockingJobQueue._wait_for_room()
        • BlockingJobQueue.wait_until_finished()
        • BlockingJobQueue.submit()
      • _DelayedFuture
        • _DelayedFuture.result()
      • _DelayedBlockingJobQueue
        • _DelayedBlockingJobQueue.submit()
        • _DelayedBlockingJobQueue._submit_if_room()
        • _DelayedBlockingJobQueue._job_result_accessed_callback()
      • _MaxQueuePool
        • _MaxQueuePool.submit()
        • _MaxQueuePool.pool_queue_callback()
        • _MaxQueuePool.shutdown()
      • coerce_num_workers()
    • kwutil.util_path module
      • tree()
      • coerce_patterned_paths()
      • find()
      • resolve_relative_to()
      • resolve_directory_symlinks()
      • sidecar_glob()
      • sanitize_path_name()
    • kwutil.util_pattern module
      • PatternBase
        • PatternBase.match()
        • PatternBase.search()
        • PatternBase.sub()
      • _maybe_expandable_glob()
      • Pattern
        • Pattern.to_regex()
        • Pattern.from_regex()
        • Pattern.from_glob()
        • Pattern._prefix_mappings
        • Pattern.coerce_backend()
        • Pattern.coerce()
        • Pattern.match()
        • Pattern.search()
        • Pattern.sub()
        • Pattern.paths()
      • MultiPattern
        • MultiPattern.matches()
        • MultiPattern.match()
        • MultiPattern.paths()
        • MultiPattern._squeeze()
        • MultiPattern.coerce()
      • FakeParseModule
        • FakeParseModule.Parser()
    • kwutil.util_progress module
      • ProgIter2
        • ProgIter2._set_manager()
        • ProgIter2.update_info()
        • ProgIter2.update()
        • ProgIter2.display_message()
        • ProgIter2.set_postfix()
      • RichProgIter
        • RichProgIter.start()
        • RichProgIter.stop()
        • RichProgIter.begin()
        • RichProgIter.end()
        • RichProgIter.update()
        • RichProgIter.step()
        • RichProgIter.remove()
        • RichProgIter.update_info()
        • RichProgIter.ensure_newline()
        • RichProgIter.set_postfix_str()
        • RichProgIter.set_postfix()
        • RichProgIter.set_extra()
      • BaseProgIterManager
        • BaseProgIterManager.new()
        • BaseProgIterManager.__call__()
        • BaseProgIterManager.start()
        • BaseProgIterManager.begin()
        • BaseProgIterManager.stop()
      • _RichProgIterManager
        • _RichProgIterManager.progiter()
        • _RichProgIterManager.ProgIter()
        • _RichProgIterManager.setup_rich()
        • _RichProgIterManager.update_info()
        • _RichProgIterManager.start()
        • _RichProgIterManager.stop()
      • _ProgIterManager
        • _ProgIterManager.progiter()
        • _ProgIterManager.update_info()
      • ProgressManager
        • ProgressManager.progiter()
        • ProgressManager.ProgIter()
        • ProgressManager.update_info()
        • ProgressManager.start()
        • ProgressManager.stop()
        • ProgressManager._is_main_manager
        • ProgressManager.stopall()
      • _progman_test_multiple_managers()
      • _progman_test_multiple_managers_tree()
    • kwutil.util_prompt module
      • PromptAlarmInterrupt
      • handle_alarm()
      • confirm_with_timeout()
    • kwutil.util_random module
      • _npstate_to_pystate()
      • _pystate_to_npstate()
      • _coerce_rng_type()
      • ensure_rng()
    • kwutil.util_resources module
      • request_nofile_limits()
      • get_shm_size()
      • check_shm_limits()
    • kwutil.util_rich module
      • _tryimport_rich()
      • _get_rich_print()
      • rich_print()
    • kwutil.util_time module
    • kwutil.util_units module
      • unit_registry()
    • kwutil.util_windows module
      • fix_msys_path()
      • is_windows_path()
    • kwutil.util_yaml module
      • _YamlRepresenter
        • _YamlRepresenter.str_presenter()
      • _custom_ruaml_loader()
      • _custom_ruaml_dumper()
      • _custom_pyaml_dumper()
      • _custom_new_ruaml_yaml_obj()
      • Yaml
        • Yaml.dumps()
        • Yaml.load()
        • Yaml.loads()
        • Yaml.coerce()
        • Yaml.InlineList()
        • Yaml.Dict()
        • Yaml.CodeBlock()
  • Module contents
    • The kwutil Module
    • CopyManager
      • CopyManager._operation_name
      • CopyManager._unsubmitted_report()
      • CopyManager._worker_func()
      • CopyManager.report()
      • CopyManager.submit()
    • DeleteManager
      • DeleteManager._operation_name
      • DeleteManager._worker_func()
      • DeleteManager.submit()
      • DeleteManager.submit_many()
    • DotDict
      • DotDict._trie_cache
      • DotDict.from_nested()
      • DotDict.insert_prefix()
      • DotDict.prefix_subdict()
      • DotDict.query_keys()
      • DotDict.suffix_subdict()
      • DotDict.to_nested()
    • Hardware
      • Hardware.cpus()
      • Hardware.disks()
      • Hardware.gpus()
      • Hardware.memory()
      • Hardware.motherboard()
      • Hardware.networking()
      • Hardware.peripherals()
      • Hardware.report()
    • Json
      • Json._load_filepointer()
      • Json.coerce()
      • Json.debug_unserializable()
      • Json.dump()
      • Json.dumps()
      • Json.ensure_serializable()
      • Json.find_unserializable()
      • Json.load()
      • Json.loads()
    • MoveManager
      • MoveManager._check()
      • MoveManager._operation_name
      • MoveManager._worker_func()
      • MoveManager.submit()
    • MultiPattern
      • MultiPattern._squeeze()
      • MultiPattern.coerce()
      • MultiPattern.match()
      • MultiPattern.matches()
      • MultiPattern.paths()
    • Pattern
      • Pattern._prefix_mappings
      • Pattern.coerce()
      • Pattern.coerce_backend()
      • Pattern.from_glob()
      • Pattern.from_regex()
      • Pattern.match()
      • Pattern.paths()
      • Pattern.search()
      • Pattern.sub()
      • Pattern.to_regex()
    • ProcessContext
      • ProcessContext.__call__()
      • ProcessContext._cpuinfo()
      • ProcessContext._device_info()
      • ProcessContext._flush_emissions_tracker()
      • ProcessContext._gpuinfo()
      • ProcessContext._hostinfo()
      • ProcessContext._infer_dynamic_properties()
      • ProcessContext._infer_static_properties()
      • ProcessContext._machine()
      • ProcessContext._meminfo()
      • ProcessContext._osinfo()
      • ProcessContext._pyinfo()
      • ProcessContext._start_emissions_tracker()
      • ProcessContext._stop_emissions_tracker()
      • ProcessContext._timestamp()
      • ProcessContext.add_device_info()
      • ProcessContext.add_disk_info()
      • ProcessContext.dump()
      • ProcessContext.flush()
      • ProcessContext.is_running
      • ProcessContext.is_started
      • ProcessContext.start()
      • ProcessContext.stop()
      • ProcessContext.write_invocation()
    • ProgressManager
      • ProgressManager.ProgIter()
      • ProgressManager._is_main_manager
      • ProgressManager.progiter()
      • ProgressManager.start()
      • ProgressManager.stop()
      • ProgressManager.stopall()
      • ProgressManager.update_info()
    • Superlock
      • Superlock.GLOBAL_APPNAME
      • Superlock.GLOBAL_LOCK_FNAME
      • Superlock.GLOBAL_THREAD_KEY
      • Superlock.THREAD_LOCKS
      • Superlock._debug()
      • Superlock.acquire()
      • Superlock.global_lock_fpath
      • Superlock.release()
    • XML
      • XML.dump()
      • XML.dumps()
      • XML.load()
      • XML.loads()
    • Yaml
      • Yaml.CodeBlock()
      • Yaml.Dict()
      • Yaml.InlineList()
      • Yaml.coerce()
      • Yaml.dumps()
      • Yaml.load()
      • Yaml.loads()
    • coerce_num_workers()
    • ensure_rng()
    • envflag()
    • safeeval()
    • sanitize_path_name()
• kwutil
  • kwutil package
    • Submodules
      • kwutil.copy_manager module
        • CopyManager
      • kwutil.fsops_managers module
        • _FilesystemOperationManager
        • _copy_worker()
        • _move_worker()
        • DeleteManager
        • CopyManager
        • MoveManager
        • remove_empty_dirs()
        • _unsubmitted_info()
        • _pathinfo()
      • kwutil.partial_format module
        • partial_format()
        • _PartialFormatter
        • _test_partial_format()
        • subtemplate()
        • fsubtemplate()
        • _get_stack_frame()
      • kwutil.process_context module
        • ProcessContext
        • jsonify_config()
        • Reconstruction
        • main()
      • kwutil.slugify_ext module
        • slugify()
        • smart_truncate()
      • kwutil.util_environ module
        • envflag()
      • kwutil.util_eval module
        • RestrictedSyntaxError
        • restricted_eval()
        • EvalException
        • ValidationException
        • CompilationException
        • ExecutionException
        • SafeAST
        • evalidate()
        • safeeval()
      • kwutil.util_exception module
        • add_exception_note()
      • kwutil.util_hardware module
        • get_cpu_mem_info()
        • get_cpu_info()
        • get_mem_info()
        • disk_info_of_path()
        • _device_is_hdd()
        • _zfs_status()
        • _torch_device_info()
        • _HardwareComponent
        • CPUs
        • Memory
        • GPUs
        • Motherboard
        • Disks
        • Networking
        • Peripherals
        • _hardware_nested_system_info()
        • _tup_to_nested()
        • Hardware
      • kwutil.util_json module
        • debug_json_unserializable()
        • ensure_json_serializable()
        • find_json_unserializable()
        • Json
      • kwutil.util_locks module
        • Superlock
      • kwutil.util_parallel module
        • BlockingJobQueue
        • _DelayedFuture
        • _DelayedBlockingJobQueue
        • _MaxQueuePool
        • coerce_num_workers()
      • kwutil.util_path module
        • tree()
        • coerce_patterned_paths()
        • find()
        • resolve_relative_to()
        • resolve_directory_symlinks()
        • sidecar_glob()
        • sanitize_path_name()
      • kwutil.util_pattern module
        • PatternBase
        • _maybe_expandable_glob()
        • Pattern
        • MultiPattern
        • FakeParseModule
      • kwutil.util_progress module
        • ProgIter2
        • RichProgIter
        • BaseProgIterManager
        • _RichProgIterManager
        • _ProgIterManager
        • ProgressManager
        • _progman_test_multiple_managers()
        • _progman_test_multiple_managers_tree()
      • kwutil.util_prompt module
        • PromptAlarmInterrupt
        • handle_alarm()
        • confirm_with_timeout()
      • kwutil.util_random module
        • _npstate_to_pystate()
        • _pystate_to_npstate()
        • _coerce_rng_type()
        • ensure_rng()
      • kwutil.util_resources module
        • request_nofile_limits()
        • get_shm_size()
        • check_shm_limits()
      • kwutil.util_rich module
        • _tryimport_rich()
        • _get_rich_print()
        • rich_print()
      • kwutil.util_time module
      • kwutil.util_units module
        • unit_registry()
      • kwutil.util_windows module
        • fix_msys_path()
        • is_windows_path()
      • kwutil.util_yaml module
        • _YamlRepresenter
        • _custom_ruaml_loader()
        • _custom_ruaml_dumper()
        • _custom_pyaml_dumper()
        • _custom_new_ruaml_yaml_obj()
        • Yaml
    • Module contents
      • The kwutil Module
      • CopyManager
        • CopyManager._operation_name
        • CopyManager._unsubmitted_report()
        • CopyManager._worker_func()
        • CopyManager.report()
        • CopyManager.submit()
      • DeleteManager
        • DeleteManager._operation_name
        • DeleteManager._worker_func()
        • DeleteManager.submit()
        • DeleteManager.submit_many()
      • DotDict
        • DotDict._trie_cache
        • DotDict.from_nested()
        • DotDict.insert_prefix()
        • DotDict.prefix_subdict()
        • DotDict.query_keys()
        • DotDict.suffix_subdict()
        • DotDict.to_nested()
      • Hardware
        • Hardware.cpus()
        • Hardware.disks()
        • Hardware.gpus()
        • Hardware.memory()
        • Hardware.motherboard()
        • Hardware.networking()
        • Hardware.peripherals()
        • Hardware.report()
      • Json
        • Json._load_filepointer()
        • Json.coerce()
        • Json.debug_unserializable()
        • Json.dump()
        • Json.dumps()
        • Json.ensure_serializable()
        • Json.find_unserializable()
        • Json.load()
        • Json.loads()
      • MoveManager
        • MoveManager._check()
        • MoveManager._operation_name
        • MoveManager._worker_func()
        • MoveManager.submit()
      • MultiPattern
        • MultiPattern._squeeze()
        • MultiPattern.coerce()
        • MultiPattern.match()
        • MultiPattern.matches()
        • MultiPattern.paths()
      • Pattern
        • Pattern._prefix_mappings
        • Pattern.coerce()
        • Pattern.coerce_backend()
        • Pattern.from_glob()
        • Pattern.from_regex()
        • Pattern.match()
        • Pattern.paths()
        • Pattern.search()
        • Pattern.sub()
        • Pattern.to_regex()
      • ProcessContext
        • ProcessContext.__call__()
        • ProcessContext._cpuinfo()
        • ProcessContext._device_info()
        • ProcessContext._flush_emissions_tracker()
        • ProcessContext._gpuinfo()
        • ProcessContext._hostinfo()
        • ProcessContext._infer_dynamic_properties()
        • ProcessContext._infer_static_properties()
        • ProcessContext._machine()
        • ProcessContext._meminfo()
        • ProcessContext._osinfo()
        • ProcessContext._pyinfo()
        • ProcessContext._start_emissions_tracker()
        • ProcessContext._stop_emissions_tracker()
        • ProcessContext._timestamp()
        • ProcessContext.add_device_info()
        • ProcessContext.add_disk_info()
        • ProcessContext.dump()
        • ProcessContext.flush()
        • ProcessContext.is_running
        • ProcessContext.is_started
        • ProcessContext.start()
        • ProcessContext.stop()
        • ProcessContext.write_invocation()
      • ProgressManager
        • ProgressManager.ProgIter()
        • ProgressManager._is_main_manager
        • ProgressManager.progiter()
        • ProgressManager.start()
        • ProgressManager.stop()
        • ProgressManager.stopall()
        • ProgressManager.update_info()
      • Superlock
        • Superlock.GLOBAL_APPNAME
        • Superlock.GLOBAL_LOCK_FNAME
        • Superlock.GLOBAL_THREAD_KEY
        • Superlock.THREAD_LOCKS
        • Superlock._debug()
        • Superlock.acquire()
        • Superlock.global_lock_fpath
        • Superlock.release()
      • XML
        • XML.dump()
        • XML.dumps()
        • XML.load()
        • XML.loads()
      • Yaml
        • Yaml.CodeBlock()
        • Yaml.Dict()
        • Yaml.InlineList()
        • Yaml.coerce()
        • Yaml.dumps()
        • Yaml.load()
        • Yaml.loads()
      • coerce_num_workers()
      • ensure_rng()
      • envflag()
      • safeeval()
      • sanitize_path_name()

Indices and tables

• Index

• Module Index

• Search Page