SequentialOutputBuffer< T > Class Template Reference

#include <genesis/utils/threading/sequential_output_buffer.hpp>

Detailed Description

template<typename T>
class genesis::utils::SequentialOutputBuffer< T >

Buffer structure for output to be produced in a well-defined sequential order.

In multi-threaded processing, we might have cases where elements are computed in some semi-random order, depending on the speed of processing of each element in the compute threads. However, for ordered output, we might need the elements to be processed in their original sequential order. This class helps to achive this, by buffering elements along with their sequence id. Once the buffer is filled to some degree with a consecutive sequence of elements (we use blocks internally), the elements can be processed in the correct order by an output function.

Elements are emplaced with a sequence id indicating their desired position in the output. Each id can only be used once, and after all elements have been processed, there shall be no gaps in the ids up until the highest id that was emplaced. After that, calling close() or destructing the buffer will then flush the remaining set of elements if their block was not yet completely filled.

Example:

// Create a buffer that prints lines in the correct order.
SequentialOutputBuffer<std::string> seq_out_buff{
    [&]( std::string&& line ){
        std::cout << line << "\n";
    }
};

// Emplace elements in the buffer. This loop can be run in a thread pool,
// where each thread processed a subset of lines in some order. In that case,
// make sure to wait for all threads to finish before calling close().
for( size_t i = 0; i < num_lines; ++i ) {
    seq_out_buff.emplace( i, lines[i] );
}
seq_out_buff.close();

As we buffer all elements until they are ready, this class is meant for data that is roughly in the correct order already. If elements come in completely random order, we need to allocate the full range of memory spanning up to the differenence in the current id and the max id that was emplaced. It is hence best used in a multi-threaded setting, where some threads process elements in their order, so that the sequence of elements is only locally random, but overall monotonic.

Definition at line 97 of file sequential_output_buffer.hpp.

Public Member Functions
	SequentialOutputBuffer (SequentialOutputBuffer &&)=delete
	SequentialOutputBuffer (SequentialOutputBuffer const &)=delete
	SequentialOutputBuffer (std::function< void(T &&)> output_function, size_t first_sequence_id=0)
	Initialize a sequential output buffer with the function that is to be called for each element in the correct sequence order. More...
	~SequentialOutputBuffer ()
void	close ()
	Close the buffer, i.e., process all remaining elements. More...
void	emplace (size_t sequence_id, T &&element)
	Emplace an element in the buffer, at the given sequence_id. More...
void	emplace (size_t sequence_id, T const &element)
	Emplace an element in the buffer, at the given sequence_id. More...
SequentialOutputBuffer &	operator= (SequentialOutputBuffer &&)=delete
SequentialOutputBuffer &	operator= (SequentialOutputBuffer const &)=delete

Constructor & Destructor Documentation

SequentialOutputBuffer() [1/3]

SequentialOutputBuffer ( std::function< void(T &&)>  output_function, size_t  first_sequence_id = 0  )

inline

Initialize a sequential output buffer with the function that is to be called for each element in the correct sequence order.

The output_function is called exactly once for each emplaced element, in their order. By default, the sequence order starts at 0, unless first_sequence_id is set to some other value, and has to be dense, i.e., in the end, all sequence ids from the first one to the last have to be emplaced at some point, with no gaps.

Definition at line 142 of file sequential_output_buffer.hpp.

~SequentialOutputBuffer()

~SequentialOutputBuffer ( )

inline

Definition at line 153 of file sequential_output_buffer.hpp.

SequentialOutputBuffer() [2/3]

SequentialOutputBuffer ( SequentialOutputBuffer< T > const &  )

delete

SequentialOutputBuffer() [3/3]

SequentialOutputBuffer ( SequentialOutputBuffer< T > &&  )

delete

Member Function Documentation

close()

void close ( )

inline

Close the buffer, i.e., process all remaining elements.

After this, no new elements can be emplaced any more.

Definition at line 234 of file sequential_output_buffer.hpp.

emplace() [1/2]

void emplace ( size_t  sequence_id, T &&  element  )

inline

Emplace an element in the buffer, at the given sequence_id.

This overload places the element in the buffer by moving. Preferred if possible.

Definition at line 184 of file sequential_output_buffer.hpp.

emplace() [2/2]

void emplace ( size_t  sequence_id, T const &  element  )

inline

Emplace an element in the buffer, at the given sequence_id.

This overload places a copy of the element in the buffer.

Definition at line 173 of file sequential_output_buffer.hpp.

operator=() [1/2]

SequentialOutputBuffer& operator= ( SequentialOutputBuffer< T > &&  )

delete

operator=() [2/2]

SequentialOutputBuffer& operator= ( SequentialOutputBuffer< T > const &  )

delete

---

The documentation for this class was generated from the following file:

• sequential_output_buffer.hpp