apache_beam.runners.interactive.utils module

Utilities to be used in Interactive Beam.

apache_beam.runners.interactive.utils.to_element_list(reader, coder, include_window_info, n=None, include_time_events=False)

Returns an iterator that properly decodes the elements from the reader.

apache_beam.runners.interactive.utils.elements_to_df(elements, include_window_info=False)

Parses the given elements into a Dataframe.

If the elements are a list of WindowedValues, then it will break out the elements into their own DataFrame and return it. If include_window_info is True, then it will concatenate the windowing information onto the elements DataFrame.

apache_beam.runners.interactive.utils.register_ipython_log_handler()

Adds the IPython handler to a dummy parent logger (named ‘apache_beam.runners.interactive’) of all interactive modules’ loggers so that if is_in_notebook, logging displays the logs as HTML in frontends.

class apache_beam.runners.interactive.utils.IPythonLogHandler(level=0)

Bases: logging.Handler

A logging handler to display logs as HTML in IPython backed frontends.

Initializes the instance - basically setting the formatter to None and the filter list to empty.

log_template = '\n <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/css/bootstrap.min.css" integrity="sha384-Vkoo8x4CGsO3+Hhxv8T/Q5PaXtkKtu6ug5TOeNV6gBiFeWPGFN9MuhOf23Q9Ifjh" crossorigin="anonymous">\n <div class="alert alert-{level}">{msg}</div>'

logging_to_alert_level_map = {0: 'light', 10: 'dark', 20: 'info', 30: 'warning', 40: 'danger', 50: 'danger'}

emit(record)

acquire()

Acquire the I/O thread lock.

addFilter(filter)

Add the specified filter to this handler.

close()

Tidy up any resources used by the handler.

This version removes the handler from an internal map of handlers, _handlers, which is used for handler lookup by name. Subclasses should ensure that this gets called from overridden close() methods.

createLock()

Acquire a thread lock for serializing access to the underlying I/O.

filter(record)

Determine if a record is loggable by consulting all the filters.

The default is to allow the record to be logged; any filter can veto this and the record is then dropped. Returns a zero value if a record is to be dropped, else non-zero.

Changed in version 3.2: Allow filters to be just callables.

flush()

Ensure all logging output has been flushed.

This version does nothing and is intended to be implemented by subclasses.

format(record)

Format the specified record.

If a formatter is set, use it. Otherwise, use the default formatter for the module.

get_name()

handle(record)

Conditionally emit the specified logging record.

Emission depends on filters which may have been added to the handler. Wrap the actual emission of the record with acquisition/release of the I/O thread lock. Returns whether the filter passed the record for emission.

handleError(record)

Handle errors which occur during an emit() call.

This method should be called from handlers when an exception is encountered during an emit() call. If raiseExceptions is false, exceptions get silently ignored. This is what is mostly wanted for a logging system - most users will not care about errors in the logging system, they are more interested in application errors. You could, however, replace this with a custom handler if you wish. The record which was being processed is passed in to this method.

name

release()

Release the I/O thread lock.

removeFilter(filter)

Remove the specified filter from this handler.

setFormatter(fmt)

Set the formatter for this handler.

setLevel(level)

Set the logging level of this handler. level must be an int or a str.

set_name(name)

apache_beam.runners.interactive.utils.obfuscate(*inputs)

Obfuscates any inputs into a hexadecimal string.

class apache_beam.runners.interactive.utils.ProgressIndicator(enter_text, exit_text)

Bases: object

An indicator visualizing code execution in progress.

spinner_template = '\n <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/css/bootstrap.min.css" integrity="sha384-Vkoo8x4CGsO3+Hhxv8T/Q5PaXtkKtu6ug5TOeNV6gBiFeWPGFN9MuhOf23Q9Ifjh" crossorigin="anonymous">\n <div id="{id}" class="spinner-border text-info" role="status">\n </div>'

spinner_removal_template = '\n $("#{id}").remove();'

apache_beam.runners.interactive.utils.progress_indicated(func)

A decorator using a unique progress indicator as a context manager to execute the given function within.

apache_beam.runners.interactive.utils.as_json(func)

A decorator convert python objects returned by callables to json string.

The decorated function should always return an object parsable by json.dumps. If the object is not parsable, the str() of original object is returned instead.