This class performs log file I/O operations and JSON parsing in a background thread on behalf of a ZLogReader object. More...

#include <ZblLogReader.h>

Inheritance diagram for Zbl::ZblLogReader:

[legend]

Collaboration diagram for Zbl::ZblLogReader:

[legend]

Classes
class	ZblLogError
	A class for passing log file input error information. More...

Public Slots
void	openFile (const QString &filePath, QVariantMap contextMap=QVariantMap())
	Opens the log file in a concurrent background thread. Sends the openComplete signal when open is complete. More...

void	close ()

void	loadViewport (QObject *viewport, qint64 seekPosition, qint64 viewportSize, ZblLogSearchParams searchParams, bool loadMarkupColumn=false, QVariantMap contextMap=QVariantMap())
	Loads a viewport's log record data model with the a set of records located by a file seek index. More...

void	move (QObject *viewport, qint64 recordCount, qint64 seekIndex, bool forward, ZblLogSearchParams searchParams, bool loadMarkupColumn)
	Moves the viewport forward or backward a specific number of records. More...

void	loadRecordLinkMessageText (QObject *searchmark, QVariant links)
	Loads the message text from the log file for the specified ZLogSearch object. More...

Signals
void	openComplete (bool success, QString errorMsg, QVariantMap contextMap)
	Sent when the file open operation started by calling openFile() has completed. More...

void	loadRecordsComplete (QObject *viewport, QString errorMsg, QVariantMap contextMap)
	DEPRECATED! Don't use! More...

void	loadComplete (QObject *viewport, ZblLogSearchParams searchParams, bool endOfData, QString errorMsg, QVariantMap context)
	Sent when loading log records into a viewport by calling ZblLogReader::loadViewport() has completed. More...

void	linkMessageText (QVariant linkList)
	Sent when loadRecordLinkMessageText has completed. More...

void	notifyFileSize (qint64 fileSize)
	Periodically signals the current log file size. More...

Public Member Functions
	ZblLogReader (QObject *parent=0)

virtual	~ZblLogReader ()

Static Public Member Functions
static void	zInit (int logRecordRoleCount)
	Must be called only once before calling newLogReader();. More...

static ZblLogReader *	newLogReader ()
	creates a new ZblLogReader object and moves it to the shared background log reader thread. More...

Protected Member Functions
void	timerEvent (QTimerEvent *event)
	Periodically sends notifyFileSize signal. More...

void	parseNextFileRecord (ZLogViewport *viewport, const QByteArray &ba, qint64 seekPos, ZblLogScanner &logScanner, ZDataRow &cellBuffer, ZblLogError &logError)
	Picks the next file record off the input stream buffer, parses it and contatenates results to the data model cell output buffer. More...

void	storeViewportData (ZDataRow &cellData, ZLogViewport *viewport, bool truncate=false)
	Appends data to the back of the viewport's data model. More...

int	parseAndStoreViewportData (zRowBuffer &logRecords, zSeekBuffer &seekPositions, qint64 rowCount, ZLogViewport *viewport, ZblLogScanner &logScanner)
	Parses rows from a row buffer and prepends them to the front of the viewport's data model. More...

QString	loadRowsAhead (ZLogViewport viewport, qint64 seekPos, qint64 rowCount, bool truncate, const ZblLogSearchParams searchParams, bool loadMarkupColumn, qint64 &foundCount, qint64 &storedCount, bool &eof)
	Appends log records to the data model scanning foreward in the file from the specified seek position. More...

QString	loadRowsBehind (ZLogViewport viewport, qint64 seekPos, qint64 rowCount, const ZblLogSearchParams searchParams, bool loadMarkupColumn, qint64 &foundCount, qint64 &storedCount, bool &bof)
	Prepends log data records to the data model scanning backward in the file from the specified seek position. More...

void	initializeMetadata ()
	Reads the log file metadata contained in the first record of the log file. More...

Protected Attributes
ZBL_DECLARE_LOGGED_OBJECT QFile	m_inputFile
	Input file object from which log records will be read. More...

bool	m_fileActive
	true if input file is open and ready for reading, false otherwise. More...

ZSettings *	m_notationFile
	a file for storing metadata (bookmarks, notations, identification) about the log file. More...

QVariantMap	m_metadata
	A QVariantMap containing meta information about the log file. More...

QUuid	m_uuid
	A UUID that uniquely identifies this log file. More...

qint64	m_firstRecordPos
	File position of the first log record of the search operation. More...

int	m_timerID
	Timer ID for file size timer. Zero == timer inactive. More...

Static Protected Attributes
static QMutex	m_lock
	Histogram buffer caches bytes of file data for subsequent parsing into log records. More...

static QThread *	m_readerThread = 0
	A single background thread services all ZblLogReader objects. More...

static int	m_readerCount = 0
	Counts instances of ZblLogReader objects. More...

static qint64	m_defaultPageSize = 10000
	number of rows in a default page. More...

static int	m_recordRoleCount = -1
	Number of roles in a log record. More...

static const int	m_updateBatchSize = 10
	The number of rows to accumulate before sending a row update to the data model. Batching rows reduces thread context switching overhead. More...

static const qint64	m_maxLineSize = 20000
	The maximum number of bytes a log record can contain. More...

Private Types
typedef QQueue< QByteArray >	zRowBuffer

typedef QList< qint64 >	zSeekBuffer

Detailed Description

This class performs log file I/O operations and JSON parsing in a background thread on behalf of a ZLogReader object.

A single background thread supports all ZblLogReader objects. Log files are read into ZblTableModel objects that can be displayed by QML container classes. Each model is encapsulated in a ZLogViewport object that contains a subset of the log file's records.

Viewports can be scrolled forward or backward and they can be resized and relocated.

Zuble log files contain a File Identification Record (FIR) as the first record of the log file (seek position zero up to the first line feed) with an identifying UUID and JSON metadata object. The UUID links the log file to it's associated notation file.

A File Identification Record is encoded as follows:

{xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx}{JSON-OBJECT}

Where {xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx} represents a UUID that uniquely identifies the log file.

Where {JSON-OBJECT} represents a JSON formatted object containing at least the following keys:

zuble-version
log-format-version
app-name
app-version
creation-date
creation-time

TBD: rename these keys to change dash to underscore

Log file consumers should ignore keys they don't recognize. Log file producers are responsible for ensuring the uniqueness of the encapsulated UUID.

Definition at line 91 of file ZblLogReader.h.

Member Typedef Documentation

◆ zRowBuffer

typedef QQueue<QByteArray> Zbl::ZblLogReader::zRowBuffer

private

Definition at line 95 of file ZblLogReader.h.

◆ zSeekBuffer

typedef QList<qint64> Zbl::ZblLogReader::zSeekBuffer

private

Definition at line 96 of file ZblLogReader.h.

Constructor & Destructor Documentation

◆ ZblLogReader()

Zbl::ZblLogReader::ZblLogReader ( QObject * parent = 0 )

explicit

Definition at line 58 of file ZblLogReader.cpp.

Referenced by newLogReader().

◆ ~ZblLogReader()

Zbl::ZblLogReader::~ZblLogReader ( )

virtual

Definition at line 69 of file ZblLogReader.cpp.

Member Function Documentation

◆ close

void Zbl::ZblLogReader::close ( )

slot

Definition at line 231 of file ZblLogReader.cpp.

Referenced by openFile().

◆ initializeMetadata()

void Zbl::ZblLogReader::initializeMetadata ( )

protected

Reads the log file metadata contained in the first record of the log file.

Metadata is stored in the m_fileMetadata variable. This method sets the m_firstRecordPos variable to the seek position of the first log record.

Returns: true if metadata was read successfully, false otherwise.

Definition at line 91 of file ZblLogReader.cpp.

Referenced by openFile().

◆ linkMessageText

void Zbl::ZblLogReader::linkMessageText ( QVariant linkList )

signal

Sent when loadRecordLinkMessageText has completed.

Parameters

linkList An encapsulated ZblLogLinkList object containing message text.

◆ loadComplete

void Zbl::ZblLogReader::loadComplete	(	QObject *	viewport,
		ZblLogSearchParams	searchParams,
		bool	endOfData,
		QString	errorMsg,
		QVariantMap	context
	)

signal

Sent when loading log records into a viewport by calling ZblLogReader::loadViewport() has completed.

Parameters

viewport	The ZLogViewport object
searchParams	search parameters that were passed to loadViewport() method
endOfData	true if load operation encountered end-of-data, false otherwise
errorMsg	a message pertaining to an error condition or empty string if no errors
context	A user-defined QVariantMap that was passed to the ZblLogReader::loadViewport()

Referenced by loadViewport().

◆ loadRecordLinkMessageText

void Zbl::ZblLogReader::loadRecordLinkMessageText	(	QObject *	searchmark,
		QVariant	links
	)

slot

Loads the message text from the log file for the specified ZLogSearch object.

This method should be invoked from the calling thread with a queued connection type so that it operates asynchronously.

The onLinkTextComplete slot of the specified searchmark object is called asynchronously (with a queued connection) when the load operation has completed.

Parameters

searchmark	A ZblLogSearch object for which link messages will be loaded
links	An encapsulated ZblLogLinkList object containing the log links to be loaded. For each link the index is the log record's ID and the value is its file seek position.

Definition at line 816 of file ZblLogReader.cpp.

◆ loadRecordsComplete

void Zbl::ZblLogReader::loadRecordsComplete	(	QObject *	viewport,
		QString	errorMsg,
		QVariantMap	contextMap
	)

signal

DEPRECATED! Don't use!

Parameters

viewport
errorMsg
contextMap

◆ loadRowsAhead()

QString Zbl::ZblLogReader::loadRowsAhead	(	ZLogViewport *	viewport,
		qint64	seekPos,
		qint64	rowCount,
		bool	truncate,
		const ZblLogSearchParams *	searchParams,
		bool	loadMarkupColumn,
		qint64 &	foundCount,
		qint64 &	storedCount,
		bool &	eof
	)

protected

Appends log records to the data model scanning foreward in the file from the specified seek position.

Parameters

viewport	Viewport into which data will be loaded from file
seekPos	starting file position
rowCount	number of log records to load
truncate	true to remove an equal number of rows from the beginning of the data model as were appended to the end
searchParams	search parameters
foundCount	number of complete data rows found in file
storedCount	number of log records parsed and stored in model
eof	true if beginning of file was encountered during load operation, false if not

Returns: error message if an error occurred, empty string if no errors encountered

Definition at line 311 of file ZblLogReader.cpp.

Referenced by loadViewport(), and move().

◆ loadRowsBehind()

QString Zbl::ZblLogReader::loadRowsBehind	(	ZLogViewport *	viewport,
		qint64	seekPos,
		qint64	rowCount,
		const ZblLogSearchParams *	searchParams,
		bool	loadMarkupColumn,
		qint64 &	foundCount,
		qint64 &	storedCount,
		bool &	bof
	)

protected

Prepends log data records to the data model scanning backward in the file from the specified seek position.

Parameters

viewport	Viewport into which data will be loaded from file
seekPos	starting file position
rowCount	number of log records to load
searchParams	search parameters
foundCount	number of complete data rows found in file
storedCount	number of log records parsed and stored in model
bof	true if beginning of file was encountered during load operation, false if not

Returns: error message if an error occurred, empty string if no errors encountered

Definition at line 623 of file ZblLogReader.cpp.

Referenced by move().

◆ loadViewport

void Zbl::ZblLogReader::loadViewport	(	QObject *	viewport,
		qint64	seekPosition,
		qint64	viewportSize,
		ZblLogSearchParams	searchParams,
		bool	loadMarkupColumn = `false`,
		QVariantMap	contextMap = `QVariantMap()`
	)

slot

Loads a viewport's log record data model with the a set of records located by a file seek index.

This method should be invoked from the calling thread with a queued connection type so that it operates asynchronously.

This is an asynchronous operation that sends the loadComplete signal when the operation has completed or failed. This signal is sent on both the ZblLogReader and ZLogViewport objects

Parameters

viewport	The ZLogViewport object to load with log records
seekPosition	The file seek position to start reading
viewportSize	The number of log records to load.
searchParams	A ZblSearchParams object to use for highlighting, or an invalid QVariant for no highlighting
loadMarkupColumn	true to load data into markup column of log record data model, false to ignore markup column
contextMap	A user-defined variant map that is passed back with the loadRecordsComplete signal

Definition at line 246 of file ZblLogReader.cpp.

◆ move

void Zbl::ZblLogReader::move	(	QObject *	viewport,
		qint64	recordCount,
		qint64	seekIndex,
		bool	forward,
		ZblLogSearchParams	searchParams,
		bool	loadMarkupColumn
	)

slot

Moves the viewport forward or backward a specific number of records.

Parameters

viewport	The viewport to move, must be a ZLogViewport object
recordCount	the number of records to move
seekIndex	current file seek position of first record in viewport if forward == false; current file seek position of first record after viewport if forward == true;
forward	true to move forward, false to move backward
searchParams	A ZblSearchParams object to use for scanning/highlighting, or an invalid ZblSearchParams object to disable scanning/highlighting
loadMarkupColumn	true to load data into markup column of viewport's log record data model, false to ignore markup column. If loadMarkupColumn == true and searchParams is valid scanning/highlighting will occur.

Definition at line 755 of file ZblLogReader.cpp.

◆ newLogReader()

ZblLogReader * Zbl::ZblLogReader::newLogReader ( )

static

creates a new ZblLogReader object and moves it to the shared background log reader thread.

Returns: A pointer to the new ZblLogReader object. Clients should only make queued connections to this object because it is owned by the background log reader thread.

Definition at line 180 of file ZblLogReader.cpp.

Referenced by Zbl::ZLogReader::ZLogReader().

◆ notifyFileSize

void Zbl::ZblLogReader::notifyFileSize ( qint64 fileSize )

signal

Periodically signals the current log file size.

Parameters

fileSize

Referenced by timerEvent().

◆ openComplete

void Zbl::ZblLogReader::openComplete	(	bool	success,
		QString	errorMsg,
		QVariantMap	contextMap
	)

signal

Sent when the file open operation started by calling openFile() has completed.

Parameters

success	true if file opened successully, false if open failed
errorMsg	An error message if open failed
contextMap	A user-defined context that was passed to the

Referenced by openFile().

◆ openFile

void Zbl::ZblLogReader::openFile	(	const QString &	filePath,
		QVariantMap	contextMap = `QVariantMap()`
	)

slot

Opens the log file in a concurrent background thread. Sends the openComplete signal when open is complete.

Parameters

filePath Path to a Zuble log file

Definition at line 201 of file ZblLogReader.cpp.

◆ parseAndStoreViewportData()

int Zbl::ZblLogReader::parseAndStoreViewportData	(	zRowBuffer &	logRecords,
		zSeekBuffer &	seekPositions,
		qint64	rowCount,
		ZLogViewport *	viewport,
		ZblLogScanner &	logScanner
	)

protected

Parses rows from a row buffer and prepends them to the front of the viewport's data model.

Parameters

logRecords	a list of byte arrays, one per row
rowCount	the number of rows to parse & store
viewport	the viewport in which to store the rows

Returns: the number of rows that were stored

Definition at line 534 of file ZblLogReader.cpp.

Referenced by loadRowsBehind().

◆ parseNextFileRecord()

void Zbl::ZblLogReader::parseNextFileRecord	(	ZLogViewport *	viewport,
		const QByteArray &	ba,
		qint64	seekPos,
		ZblLogScanner &	logScanner,
		ZDataRow &	cellBuffer,
		ZblLogError &	logError
	)

protected

Picks the next file record off the input stream buffer, parses it and contatenates results to the data model cell output buffer.

Parameters

viewport
ba
seekPos
logScanner
cellBuffer
logError

Definition at line 446 of file ZblLogReader.cpp.

Referenced by loadRowsAhead(), and parseAndStoreViewportData().

◆ storeViewportData()

void Zbl::ZblLogReader::storeViewportData	(	ZDataRow &	cellData,
		ZLogViewport *	viewport,
		bool	truncate = `false`
	)

protected

Appends data to the back of the viewport's data model.

Parameters

cellData	data to store
viewport	viewport in which to store data
truncate	true to remove an equal amount of rows from the model as those that were stored.

Definition at line 524 of file ZblLogReader.cpp.

Referenced by loadRowsAhead().

◆ timerEvent()

void Zbl::ZblLogReader::timerEvent ( QTimerEvent * event )

protected

Periodically sends notifyFileSize signal.

Parameters

event

Definition at line 168 of file ZblLogReader.cpp.

◆ zInit()

void Zbl::ZblLogReader::zInit ( int logRecordRoleCount )

static

Must be called only once before calling newLogReader();.

Parameters

logRecordRoleCount Number of roles in a log record.

Definition at line 75 of file ZblLogReader.cpp.

Referenced by Zbl::ZLogReader::registerType().

Member Data Documentation

◆ m_defaultPageSize

qint64 Zbl::ZblLogReader::m_defaultPageSize = 10000

staticprotected

number of rows in a default page.

Definition at line 502 of file ZblLogReader.h.

◆ m_fileActive

bool Zbl::ZblLogReader::m_fileActive

protected

true if input file is open and ready for reading, false otherwise.

Definition at line 467 of file ZblLogReader.h.

Referenced by close(), openFile(), and timerEvent().

◆ m_firstRecordPos

qint64 Zbl::ZblLogReader::m_firstRecordPos

protected

File position of the first log record of the search operation.

Definition at line 553 of file ZblLogReader.h.

Referenced by initializeMetadata().

◆ m_inputFile

ZBL_DECLARE_LOGGED_OBJECT QFile Zbl::ZblLogReader::m_inputFile

protected

Input file object from which log records will be read.

Definition at line 461 of file ZblLogReader.h.

Referenced by close(), initializeMetadata(), loadRecordLinkMessageText(), loadRowsAhead(), loadRowsBehind(), loadViewport(), openFile(), and timerEvent().

◆ m_lock

QMutex Zbl::ZblLogReader::m_lock

staticprotected

Histogram buffer caches bytes of file data for subsequent parsing into log records.

The page map caches the seek location of pages already scanned to speed up subsequent viewport operations. locks the thread and instance counter objects.

Definition at line 486 of file ZblLogReader.h.

Referenced by newLogReader().

◆ m_maxLineSize

const qint64 Zbl::ZblLogReader::m_maxLineSize = 20000

staticprotected

The maximum number of bytes a log record can contain.

Definition at line 520 of file ZblLogReader.h.

Referenced by initializeMetadata(), loadRecordLinkMessageText(), loadRowsAhead(), and loadRowsBehind().

◆ m_metadata

QVariantMap Zbl::ZblLogReader::m_metadata

protected

A QVariantMap containing meta information about the log file.

This object is stored as a JSON object in the log file's first record, after the log file's identifying UUID

Definition at line 536 of file ZblLogReader.h.

Referenced by initializeMetadata().

◆ m_notationFile

ZSettings* Zbl::ZblLogReader::m_notationFile

protected

a file for storing metadata (bookmarks, notations, identification) about the log file.

Definition at line 527 of file ZblLogReader.h.

◆ m_readerCount

int Zbl::ZblLogReader::m_readerCount = 0

staticprotected

Counts instances of ZblLogReader objects.

Definition at line 497 of file ZblLogReader.h.

Referenced by newLogReader().

◆ m_readerThread

QThread * Zbl::ZblLogReader::m_readerThread = 0

staticprotected

A single background thread services all ZblLogReader objects.

Definition at line 491 of file ZblLogReader.h.

Referenced by newLogReader().

◆ m_recordRoleCount

int Zbl::ZblLogReader::m_recordRoleCount = -1

staticprotected

Number of roles in a log record.

Definition at line 508 of file ZblLogReader.h.

Referenced by loadRecordLinkMessageText(), parseNextFileRecord(), and zInit().

◆ m_timerID

int Zbl::ZblLogReader::m_timerID

protected

Timer ID for file size timer. Zero == timer inactive.

Definition at line 559 of file ZblLogReader.h.

Referenced by close(), and openFile().

◆ m_updateBatchSize

const int Zbl::ZblLogReader::m_updateBatchSize = 10

staticprotected

The number of rows to accumulate before sending a row update to the data model. Batching rows reduces thread context switching overhead.

Definition at line 514 of file ZblLogReader.h.

Referenced by loadRowsAhead().

◆ m_uuid

QUuid Zbl::ZblLogReader::m_uuid

protected

A UUID that uniquely identifies this log file.

This ID is used to link Zuble log files to their corresponding log notation files. This value is stored as GUID-formatted hex string at file seek position zero. It is immediately followed in the log file by a JSON object containing the log file's metadata if that information is available.

Definition at line 547 of file ZblLogReader.h.

Referenced by initializeMetadata().

The documentation for this class was generated from the following files:

zblcore/ZblLogReader.h
zblcore/ZblLogReader.cpp

Classes

Public Slots

Signals

Public Member Functions

Static Public Member Functions

Protected Member Functions

Protected Attributes

Static Protected Attributes

Private Types

Detailed Description

Member Typedef Documentation

◆ zRowBuffer

◆ zSeekBuffer

Constructor & Destructor Documentation

◆ ZblLogReader()

◆ ~ZblLogReader()

Member Function Documentation

◆ close

◆ initializeMetadata()

◆ linkMessageText

◆ loadComplete

◆ loadRecordLinkMessageText

◆ loadRecordsComplete

◆ loadRowsAhead()

◆ loadRowsBehind()

◆ loadViewport

◆ move

◆ newLogReader()

◆ notifyFileSize

◆ openComplete

◆ openFile

◆ parseAndStoreViewportData()

◆ parseNextFileRecord()

◆ storeViewportData()

◆ timerEvent()

◆ zInit()

Member Data Documentation

◆ m_defaultPageSize

◆ m_fileActive

◆ m_firstRecordPos

◆ m_inputFile

◆ m_lock

◆ m_maxLineSize

◆ m_metadata

◆ m_notationFile

◆ m_readerCount

◆ m_readerThread

◆ m_recordRoleCount

◆ m_timerID

◆ m_updateBatchSize

◆ m_uuid