Transformation Functions

Generic reusable functions to apply transformations to event dictionaries.

Add Column Based On Null

managealooma.transformation_functions.add_column_based_on_null(event, field, new_field, new_value_if_null, new_value_if_not_null)

Checks and adds a value to a new field based on NULL

Parameters:

• event – A dictionary
• field – The name of the field to check
• new_field – The name of the new field
• new_value_if_null – The value for the new_field if field IS NULL
• new_value_if_not_null – The value for the new_field if field IS NOT NULL

Returns:

An altered dictionary

Examples:

# Example #1
event = {'do i have digits?': '1234'}
event = add_column_based_on_null(event, field='do i have digits?', new_field='digits', new_value_if_null='N', new_value_if_not_null='Y')
event = {'digits': 'Y',
        'do i have digits?': '1234'}

# Example #2
event = {'do i have digits?': None}
event = add_column_based_on_null(event, field='do i have digits?', new_field='digits', new_value_if_null='N', new_value_if_not_null='Y')
event = {'digits': 'N',
        'do i have digits?': None}

Add Columns with Default

managealooma.transformation_functions.add_columns_with_default(event, field_and_default_dict)

Adds a column with the default value to every event where it’s not already present

Parameters:

• event – A dictionary
• field_and_default_dict – A dictionary with keys and default values {field_1: default_for_field_1, field_2: default_for_field_2}

Returns:

An altered dictionary

Examples:

# Example #1
event = {'an existing column': 'some value'}
event = add_columns_with_default(event, field_and_default_dict={"new column 1": 'my default of 1', "new column two": 2})
event = {'an existing column': 'some value',
        'new column 1': 'my default of 1',
        'new column two': 2}

# Example #2
event = {'an existing column': 'some value'}
event = add_columns_with_default(event, field_and_default_dict={"mark_for_delete": False})
event = {'an existing column': 'some value',
        'mark_for_delete': False}

Add Composite Key

managealooma.transformation_functions.add_composite_key(event, field_list, key_name)

Creates a composite key to be used for a unique ID.

Parameters:

• event – A dictionary
• field_list – A list of fields to combine to make the key
• key_name – The name of the new key field

Returns:

An altered dictionary

Examples:

# Example #1
event = {'a_field': 1234,
        'a_second_field': '2019-01-01',
        'a_third_field': 'abc'}
event = add_composite_key(event, field_list=['a_field', 'a_second_field', 'a_third_field'], key_name='my_derived_key')
event = {'a_field': 1234,
        'a_second_field': '2019-01-01',
        'a_third_field': 'abc',
        'my_derived_key':
        '1234-2019-01-01-abc'}

Add Duplicate Fields

managealooma.transformation_functions.add_duplicate_fields(event, field_name, suffix_or_suffix_list, keep_original=False)

Add duplicate values of a field with a suffix

Parameters:

• event – A dictionary
• field_name – The name of the field to duplicate
• suffix_or_suffix_list – A single or list of suffixes to add to the field_name in the duplicates
• keep_original – True to keep the original field also

Returns:

An altered dictionary

Examples:

# Example #1
event = {'a_field': 'a_value'}
event = add_duplicate_fields(event, field_name='a_field', suffix_or_suffix_list='my_suffix', keep_original=False)
event = {'a_field_my_suffix': 'a_value'}

# Example #2
event = {'a_field': 'a_value'}
event = add_duplicate_fields(event, field_name='a_field', suffix_or_suffix_list='my_suffix', keep_original=True)
event = {'a_field': 'a_value',
        'a_field_my_suffix': 'a_value'}

# Example #3
event = {'a_field': 'a_value'}
event = add_duplicate_fields(event, field_name='a_field', suffix_or_suffix_list=['my_suffix','my_second_suffix'], keep_original=False)
event = {'a_field_my_second_suffix': 'a_value',
        'a_field_my_suffix': 'a_value'}

Add Suffix

managealooma.transformation_functions.add_suffix(event, fields, suffix, separator='_')

Adds a suffix to a field or list of fields

Parameters:

• event – A dict with the entire event
• field_or_field_list – A single field or list of fields for which to add a suffix
• suffix – The suffix to add to the fields
• separator – The character to place between the name and the suffix

Returns:

An altered event with the altered field names

Examples:

# Example #1
event = {'a_field': 'a_value'}
event = add_suffix(event, fields='a_field', suffix='an_ending')
event = {'a_field_an_ending': 'a_value'}

# Example #2
event = {'a_field': 'a_value'}
event = add_suffix(event, fields='a_field', suffix='an_ending', separator='---')
event = {'a_field---an_ending': 'a_value'}

# Example #3
event = {'a_field': 'a_value',
        'another_field': 'another_value'}
event = add_suffix(event, fields=['a_field', 'another_field'], suffix='an_ending')
event = {'a_field_an_ending': 'a_value',
        'another_field_an_ending': 'another_value'}

Convert All Event Fields to Snake Case

managealooma.transformation_functions.convert_all_event_fields_to_snake_case(event)

Converts all keys in an event to snake case. If a key is Partially_SnakeCase we’ll get 2 underscores where there is currently one like partially__snake_case

Parameters:event – An Alooma event Returns:A transformed event with all the keys in snake_case

Examples:

# Example #1
event = {'_metadata': {},
        'TitleCase': 'to_snake_case',
        'camelCase': 'to_snake_case',
        'snake_case': 'to_snake_case'}
event = convert_all_event_fields_to_snake_case(event)
event = {'_metadata': {},
        'camel_case': 'to_snake_case',
        'snake_case': 'to_snake_case',
        'title_case': 'to_snake_case'}

Convert Dictionary Fields to String

managealooma.transformation_functions.convert_dictionary_fields_to_string(event, field_or_field_list)

Dumps a list of fields to a string to keep Alooma from auto-parsing

Parameters:

• event – A dict with the entire event
• field_or_field_list – A single field or list of fields to json.dumps to keep Alooma from doing infinte de-nesting

Returns:

A new event

Examples:

# Example #1
event = {'a_dict': {'field_one': 1,
        'field_two': 2}}
event = convert_dictionary_fields_to_string(event, field_or_field_list='a_dict')
event = {'a_dict': '{"field_one": 1, "field_two": 2}'}

# Example #2
event = {'a_dict': {'field_one': 1,
                    'field_two': 2},
        'a_second_dict': {'field_one': 1,
                        'field_two': 2}}
event = convert_dictionary_fields_to_string(event, field_or_field_list=['a_dict', 'a_second_dict'])
event = {'a_dict': '{"field_one": 1, "field_two": 2}',
        'a_second_dict': '{"field_one": 1, "field_two": 2}'}

Convert Null to Zero

managealooma.transformation_functions.convert_null_to_zero(event, field_or_field_list)

Converts the value in a field or field list from None to 0

Parameters:

• event – a dict with the event
• field_or_field_list – A single field or list of fields to convert to 0 if null

Returns:

the updated event

Examples:

# Example #1
event = {'a_field': None}
event = convert_null_to_zero(event, field_or_field_list='a_field')
event = {'a_field': 0}

# Example #2
event = {'a_field': None,
        'another_field': None}
event = convert_null_to_zero(event, field_list=['a_field', 'another_field'])
event = {'a_field': 0,
        'another_field': 0}

Convert Spaces and Special Characters to Underscore

managealooma.transformation_functions.convert_spaces_and_special_characters_to_underscore(name)

Converts spaces and special characters to underscore so ‘Thi$ i# jun&’ becomes ‘thi__i__jun_’

Parameters:name – A string Returns:An altered string

Example use case: - A string might have special characters at the end when they are really the same field such as My_Field$ and My_Field# - We use this to covert the names to “my_field” to combine the values so the events will be easily grouped together

Examples:

# Example #1
input_string = '$Scr "get-rid^-of-the@" special #characters%&space'
output_string = convert_spaces_and_special_characters_to_underscore(input_string)
output_string = '_scr__get_rid__of_the___special__characters__space'

Convert String to Snake Case

managealooma.transformation_functions.convert_string_to_snake_case(name)

Converts a string to Snake Case

Parameters:name – A string Returns:A string in snake case

Example use case: - Events from might have custom properties in camelCase like userId, and userEmail - Use this to rename keys to user_id and user_email for better ease of reading in the database

Examples:

# Example #1
input_string = 'MakeThisSnakeCase'
output_string = convert_string_to_snake_case(input_string)
output_string = 'make_this_snake_case'

# Example #2
input_string = 'Make This Snake Case'
output_string = convert_string_to_snake_case(input_string)
output_string = 'make_this_snake_case'

# Example #3
input_string = 'keep_this_snake_case'
output_string = convert_string_to_snake_case(input_string)
output_string = 'keep_this_snake_case'

Convert Values to None

managealooma.transformation_functions.convert_values_to_none(event, field_or_field_list, field_values=None)

Changes a field to None. If a field value is specified then only that value will be changed to None

Parameters:

• event – A dictionary
• field_or_field_list – A single field or list of fields to convert to None
• field_values – The value to convert to None. If specified only these values are converted to None

Returns:

An altered dictionary

Examples:

# Example #1
event = {'a_field': 'a_value'}
event = convert_values_to_none(event, field_or_field_list='a_field')
event = {'a_field': None}

# Example #2
event = {'a_field': 'a_value',
        'another_field': 'another_value'
event = convert_values_to_none(event, field_or_field_list=['a_field', 'another_field'])
event = {'a_field': None,
        'another_field': None}

# Example #3
event = {'a_field': 'a_value',
        'another_field': 'another_value'
event = convert_values_to_none(event, fields=['a_field', 'another_field'], field_values='a_value')
event = {'a_field': None,
        'another_field': 'another_value'}

# Example #4
event = {'a_field': 'a_value',
        'another_field': 'another_value'
event = convert_values_to_none(event, fields=['a_field', 'another_field'], field_values=['a_value', 'another_value'])
event = {'a_field': None,
        'another_field': None}

Convert Empty Value to None

managealooma.transformation_functions.convert_empty_value_to_none(event, key_name)

Changes an empty string of “” or ” “, and empty list of [] or an empty dictionary of {} to None so it will be NULL in the database

Parameters:

• event – A dictionary
• key_name – The key for which to check for empty strings

Returns:

An altered dictionary

Examples:

# Example #1
event = {'a_field': ' '}
event = convert_empty_value_to_none(event, key_name='a_field')
event = {'a_field': None}

# Example #2
event = {'a_field': '{}'}
event = convert_empty_value_to_none(event, key_name='a_field')
event = {'a_field': None}

# Example #3
event = {'a_field': {}}
event = convert_empty_value_to_none(event, key_name='a_field')
event = {'a_field': None}

Convert Event Type Case

managealooma.transformation_functions.convert_event_type_case(event, case_force_upper=False)

Forces upper of lower case for event types at the end of the code engine. For Snowfalke force UPPER and for Redshift force lower.

Parameters:

• event – A dict with the entire event
• case_force_upper – True to for upper case.

Returns:

An event with the case altered in event_type

Examples:

# Example #1
event = {'_metadata': {'event_type': 'My_SCHEMA.my_table'}}
event = convert_event_type_case(event)
event = {'_metadata': {'event_type': 'my_schema.my_table'}}

# Example #2
event = {'_metadata': {'event_type': 'My_SCHEMA.my_table'}}
event = convert_event_type_case(event, case_force_upper=True)
event = {'_metadata': {'event_type': 'MY_SCHEMA.MY_TABLE'}}

Flatten JSON

managealooma.transformation_functions.flatten_json(event, field_or_field_list, levels, keep_original=False, dump_to_string=False)

Flattens a list of fields from a dictionary n levels

Parameters:

• event – the event that you want to pass through the function (formatted as a dictionary)
• field_or_field_list – A field or list of the fields that you want to flatten N levels deep
• levels – The number of levels that you want to parse the fields
• keep_original – True if you want to keep the original field in the event, false if you want to delete it

Returns:

The transformed event

Examples:

# Example #1
event = {'a_dict': {'field_one': 1,
                    'field_two': 2}}
event = flatten_json(event, field_or_field_list='a_dict', levels=1)
event = {'a_dict_field_one': 1,
        'a_dict_field_two': 2}

# Example #2
event = {'a_dict': {'field_one': 1,
                    'field_two': 2},
        'a_second_dict': {'field_one': {'one more': 1.1,
                                        'one more again': 1.2},
                        'field_two': 2}}
event = flatten_json(event, field_or_field_list=['a_dict','a_second_dict'], levels=1)
event = {'a_dict_field_one': 1,
        'a_dict_field_two': 2,
        'a_second_dict_field_one': {'one more': 1.1, 'one more again': 1.2},
        'a_second_dict_field_two': 2}

# Example #3
event = {'a_dict': {'field_one': 1,
                    'field_two': 2},
        'a_second_dict': {'field_one': {'one more': 1.1,
                                        'one more again': 1.2},
                        'field_two': 2}}
event = flatten_json(event, field_or_field_list=['a_dict','a_second_dict'], levels=2)
event = {'a_dict_field_one': 1,
        'a_dict_field_two': 2,
        'a_second_dict_field_one_one more': 1.1,
        'a_second_dict_field_one_one more again': 1.2,
        'a_second_dict_field_two': 2}

Flatten JSON 1 Level

managealooma.transformation_functions.flatten_json_1_level(event, field_name, field_name_underscore, dump_to_string)

Flattens a JSON field 1 level. This function is used in flatten JSON

Parameters:

• event – A dictionary
• field_name – The field name to flatten
• field_name_underscore – The field name with an underscore appended
• dump_to_string – If true any remaining dictionaries will be converted to a string with json.dumps

Returns:

An event with the field flattened

Examples:

# Example #1
event = {'my_field': "{"a": None, "b"}"}
event = flatten_json_1_level(event=input_event, field_name='my_field', field_name_underscore='my_field_', dump_to_string=True)
output_event = {'my_field_a': None,
                'my_field_b': 2}

Map Key in Dictionary to Value

managealooma.transformation_functions.map_key_in_dictionary_to_value(event, mapping_dict, existing_column, new_column, allow_nulls)

Adds a column mapping using a dictionary

Parameters:

• event – A dictionary
• mapping_dict – A mapping dict such as {1: ‘product A’, 2: ‘product B’}
• existing_column – The column that matches the keys in the mapping dict
• new_column – The name of the column to put the values from the mapping dict
• allow_nulls – True if the function should let a NULL value in the existing_column pass through. False to throw an error when the existing column has NULL.

Returns:

An event with the new_column k, v added

Examples:

# Example #1
event = {'a_field': 1,
        '_metadata': {'event_type': 'MY_SCHEMA.MY_TABLE'}}
event = map_key_in_dictionary_to_value(event, mapping_dict={1: 'one', 2: 'two'}, existing_column='a_field', new_column='a_mapped_field', allow_nulls=False)
event = {'_metadata': {'event_type': 'MY_SCHEMA.MY_TABLE'},
        'a_field': 1,
        'a_mapped_field': 'one'}

# Example #2
event = {'a_field': 3,
        '_metadata': {'event_type': 'MY_SCHEMA.MY_TABLE'}}
event = map_key_in_dictionary_to_value(event, mapping_dict={1: 'one', 2: 'two'}, existing_column='a_field', new_column='a_mapped_field', allow_nulls=False)
Exception: Missing enum transform MY_SCHEMA.MY_TABLE a_field

# Example #3
event = {'a_field': None,
        '_metadata': {'event_type': 'MY_SCHEMA.MY_TABLE'}}
event = map_key_in_dictionary_to_value(event, mapping_dict={1: 'one', 2: 'two'}, existing_column='a_field', new_column='a_mapped_field', allow_nulls=True)
event = {'_metadata': {'event_type': 'MY_SCHEMA.MY_TABLE'},
        'a_field': None,
        'a_mapped_field': None}

Map Value in List to Dictionary Key

managealooma.transformation_functions.map_value_in_list_to_dictionary_key(event, mapping_dict_with_lists, existing_column, new_column, allow_nulls, passthrough)

Maps a value from a list back to the key. Useful to map values to categories.

Parameters:

• event – A dictionary
• mapping_dict_with_lists – A mapping dict with lists in the values such as {“baked good”: [“cake”, “croissant”]}
• existing_column – The column with the existing data
• new_column – The name of the new column for the added data
• allow_nulls – True if the existing column can have NULL. If set to False NULLs will throw an error
• passthrough – True if we should pass through a value of the existing column when there is no mapping value in the list

Returns:

An altered event

Examples:

# Example #1
event = {'_metadata': {'event_type': 'MY_SCHEMA.MY_TABLE'},
        'a_field': 1}
event = map_value_in_list_to_dictionary_key(event, mapping_dict_with_lists={'1-3': [1, 2, 3], '4-6': [4, 5, 6]}, existing_column='a_field', new_column='a_mapped_field', allow_nulls=False, passthrough=False)
event = {'_metadata': {'event_type': 'MY_SCHEMA.MY_TABLE'},
        'a_field': 1,
        'a_mapped_field': '1-3'}

# Example #2
event = {'_metadata': {'event_type': 'MY_SCHEMA.MY_TABLE'},
        'a_field': 7}
event = map_value_in_list_to_dictionary_key(event, mapping_dict_with_lists={'1-3': [1, 2, 3], '4-6': [4, 5, 6]}, existing_column='a_field', new_column='a_mapped_field', allow_nulls=False, passthrough=False)
Exception: Missing map_list transform MY_SCHEMA.MY_TABLE a_field

Mark for Delete

managealooma.transformation_functions.mark_for_delete(event)

We created database triggers in our database to write all rows to a polymorphic deleted records table upon hard delete. We log the table_name, the time it was deleted, and the row_to_json This function creates a new row that looks like a soft delete came from the database

Parameters:event – A dictionary that includes the Alooma _metadata dictionary Returns:A dictionary that looks like a soft deleted row

Examples:

# Example #1
event = {'id': 1,
        'table_name': 'alooma_test',
        'primary_key': '123',
        'old_row_json': '{"id":6, "created_at":"2019-01-01"}',
        '_metadata': {'event_type': 'test'}}
event = mark_for_delete(event)
event = {'_metadata': {'event_type': 'alooma_test',
                        'table': 'alooma_test'},
        'created_at': '2019-01-01',
        'id': 6,
        'mark_for_delete': True}

Parse List of JSON and Concat

managealooma.transformation_functions.parse_list_of_json_and_concat(event, field_name, keep_original, field_to_keep)

Iterates through a dictionary and creates a single field with a list of values from the field Output is similar to group_concat and listagg in SQL

Parameters:

• event – A dictionary
• field_name – The name of the field to extract data from
• keep_original – True to keep the original field. False to delete the original field and only keep the parsed data.
• field_to_keep – The name of the field from within the dictionary

Returns:

An altered dictionary

Examples:

# Example #1
event = {'list_of_dicts': [{'key_to_concat': 123, 'key_to_ignore': 'abc'},
                            {'key_to_concat': 456, 'key_to_ignore': 'def'},
                            {'key_to_concat': 789, 'key_to_ignore': 'ghi'}]}
event =  parse_list_of_json_and_concat(input_event, field_name='list_of_dicts', keep_original=True, field_to_keep='key_to_concat')
event = {'list_of_dicts': [{'key_to_concat': 123, 'key_to_ignore': 'abc'},
                            {'key_to_concat': 456, 'key_to_ignore': 'def'},
                            {'key_to_concat': 789, 'key_to_ignore': 'ghi'}],
        'list_of_dicts_key_to_concats': [123, 456, 789]}

Remove Duplicate Field

managealooma.transformation_functions.remove_duplicate_field(event, field_to_keep, field_to_discard)

Remove a field when both fields are present

Parameters:

• event – A dictionary
• field_to_keep – The field to keep if both keys are present and the value is not None
• field_to_discard – The field to discard if the field_to_keep is not None

Returns:

An event with a single bundle ID

Examples:

# Example #1
event = {'A_Field': 'another_value', 'a_field': 'a_value '}
event = remove_duplicate_field(event, field_to_keep='a_field', field_to_discard='A_Field')
event = {'a_field': 'a_value '}

Remove Outer Key

managealooma.transformation_functions.remove_outer_key(event, key_name)

Removes the outer key from an event

Parameters:

• event – A dict with the entire event
• key_name – The key to remove from the dictionary

Returns:

An event with the outer key for the specified dictionary removed

Examples:

# Example #1
event = {'outer_dict': {'a_field': 'a_value ',
                        'another_field': 'another_value'}}
event = remove_outer_key(event, key_name='outer_dict')
event = {'a_field': 'a_value ',
        'another_field': 'another_value'}

Remove Starting Characters from Keys

managealooma.transformation_functions.remove_starting_characters_from_keys(event, starting_characters, field_with_json=None)

Removes the specified starting characters from all keys in an event

Parameters:

• event – A dict with the entire event
• starting_characters – The characters to remove from the beginning of the key
• field_with_json – A specific field with nested json from which to remove the characters from its keys

Returns:

a modified event

Examples:

# Example #1
event = {'_metadata': {},
        '$a_field': 'a_value',
        '$another_field': 'another_value'}
event = remove_starting_characters_from_keys(event, starting_characters='$')
event = {'_metadata': {},
        'another_field': 'another_value',
        'field': 'a_value'}

# Example #2
event = {'_metadata': {},
        'a_dict': {'$a_field': 'a_value',
                    '$another_field': 'another_value'},
        '$outer_field': 'some value'}
event = remove_starting_characters_from_keys(event, starting_characters='$', field_with_json='a_dict')
event = {'$outer_field': 'some value',
        '_metadata': {},
        'a_dict': {'a_field': 'a_value',
                    'another_field': 'another_value'}}

Remove Whitespace

managealooma.transformation_functions.remove_whitespace(event, field_or_field_list)

Remove leading and trailing whitespace

Parameters:

• event – A dictionary
• field_or_field_list – A field or list of fields to trim the whitespace from

Returns:

A trimmed string

Examples:

# Example #1
event = {'a_field': '  should not have whitespace at ends    '}
event = remove_whitespace(event, field_or_field_list='a_field')
event = {'a_field': 'should not have whitespace at ends'}

# Example #2
event = {'a_field': '  should not have whitespace at ends    ',
        'another_field': '  also should not have whitespace at ends    '}
event = remove_whitespace(event, field_or_field_list=['a_field', 'another_field'])
event = {'a_field': 'should not have whitespace at ends',
        'another_field': 'also should not have whitespace at ends'}

Rename Fields

managealooma.transformation_functions.rename_fields(event, field_dict)

Renames fields from the key to value :param event: A dict with the entire event :param field_dict: A dict with the rename mapping with the key as the existing field and the value as the new field :return: An altered event with the renamed fields

Examples:

# Example #1
event = {'a_field': 'a_value',
        'another_field': 'another_value',
        'no_change': 'same'}
event = rename_fields(event, field_dict={'a_field': 'field_one', 'another_field': 'field_two'})
event = {'field_one': 'a_value',
        'field_two': 'another_value',
        'no_change': 'same'}

Split Event to Multiple Events

managealooma.transformation_functions.split_event_to_multiple_events(event, table_name_list)

Splits events into a list of events with a schema_name.table_name

Parameters:

• event – A dict with a single event
• table_name_list – The table names for the new events

Returns:

A list of the new events. If an event has already been split it will not re-split and returns the original event.

Examples:

# Example #1
event = {'_metadata': {'@uuid': '123-abc', 'event_type':
                        'my_schema.my_table'},
        'a_field': 'a_value'}
event = split_event_to_multiple_events(event, table_name_list=['table_one', 'table_two'])
# A parent UUID is added in Alooma when events are split.  Local testing won't add a parent UUID.
event = [{'_metadata': {'@parent_uuid': '123-abc',
                        '@uuid': '456-def',
                        'event_type': 'my_schema.table_one'},
        'a_field': 'a_value'},
        {'_metadata': {'@uuid': '123-abc',
                        '@uuid': '789-ghi',
                        'event_type': 'my_schema.table_two'},
        'a_field': 'a_value'}]

# Example #2
event = {'_metadata': {'@parent_uuid': '123-abc',
                        '@uuid': '456-def',
                        'event_type': 'my_schema.table_one'},
        'a_field': 'a_value'}
event = split_event_to_multiple_events(event, table_name_list=['table_one', 'table_two'])
# If the event has a parent_uuid it will not be re-split
event = {'_metadata': {'@parent_uuid': '123-abc',
                        '@uuid': '456-def',
                        'event_type': 'my_schema.table_one'},
        'a_field': 'a_value'}

Split Field List to Multiple Events

managealooma.transformation_functions.split_field_list_to_multiple_events(event, fields_to_split, add_counter, counter_name, reverse=False)

Take an event that has columns that are lists (of same length) and break into multiple rows

Parameters:

• event – A dictionary
• fields_to_split – The field with to split
• add_counter – True to add a counter field to the event
• counter_name – The name of the counter field
• reverse – True to start the counter in reverse

Returns:

A list of events

Examples:

# Example #1
event = {'id': 1,
        'names': ['first', 'second'],
        '_metadata': {'uuid': '1a'}}
event =  split_field_list_to_multiple_events(event=input, fields_to_split=['names'], add_counter=True, counter_name='counter', reverse=False)
event = [{'id': 1,
           'name': 'first',
           'counter': 1,
           '_metadata': {'@parent_uuid': '1a',
                        '@uuid': '456-def'}},
          {'id': 1,
           'name': 'second',
           'counter': 2,
           '_metadata': {'@parent_uuid': '1a',
                        '@uuid': '789-ghi'}}]

Whitelist or Blacklist Columns

managealooma.transformation_functions.whitelist_or_blacklist_columns(event, field_list, white_or_black_list='whitelist')

Allows you to remove a list of fields (blacklist) or limit an event to a list of fields (whitelist)

Parameters:

• event – A dictionary
• field_list – A list of fields to keep or remove
• white_or_black_list – whitelist = Only let a particular list of columns through the event and remove other columns. blacklist = Don’t allow a particular list of columns through. Leave all other columns

Returns:

An altered dictionary

Examples:

# Example #1
event = {'_metadata': {},
        'keep_me': 'i stay',
        'keep_me_too': 'i stay too',
        'remove_me': 'im gone'}
event = whitelist_or_blacklist_columns(event, field_list=['keep_me', 'keep_me_too'], white_or_black_list='whitelist')
event = {'_metadata': {},
        'keep_me': 'i stay',
        'keep_me_too': 'i stay too'}

# Example #2
event = {'_metadata': {},
        'keep_me': 'i stay',
        'keep_me_too': 'i stay too',
        'remove_me': 'im gone'}
event = whitelist_or_blacklist_columns(event, field_list=['remove_me'], white_or_black_list='blacklist')
event = {'_metadata': {},
        'keep_me': 'i stay',
        'keep_me_too': 'i stay too'}