Metadata Importer

No UI

Imports metadata from a CSV file, based on an asset's path (or just the name, provided it's unique enough).

This is intended to facilitate an initial import from a 3rd party system that you want to migrate into CELUM, and you got a hard drive with the files on it and a CSV. Based on the (relative) storage path, the metadata is written to the assets in CELUM.

If you intend to use this to bulk-edit existing asset metadata, try the toolbox instead.

Properties

To be configured in {home}/appserver/conf/custom.properties

genericMappingTask.license

type: String, required: yes, default: -

The license key for the plugin (product: genericMappingTask), provided by brix.

genericMappingTask.startDate

type: Date (format: yyyy.MM.dd), required: no, default: -

Only consider assets that have been created since that date.

Bean Configuration

Because the Metadata Importer can be quite complex, the main configuration is not done through properties but rather with Spring Beans. The Metadata Mapper expects an XML to be located in {home}/appserver/spring/genericMappingTask-config.xml

Let's consider the following example where we have 3 columns: the assets path, the asset type it should get, and the creation date:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
    xmlns:util="http://www.springframework.org/schema/util"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
            http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
            http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-3.0.xsd">
    <context:annotation-config />

    <bean id="genericMappingTaskDocuments" class="ch.brix.genericMappingTask.GenericMappingTask">
        <property name="csvReader" ref="csvReader" />
        <property name="assetMappings" ref="assetMappings" />
        <property name="assetMappingFile" value="classpath:conf/import.csv" />
        <property name="assetMappingFileKeyColumn" value="0" /><!-- 0-based -->
        <property name="rootNodeId" value="11634" />
        <property name="useOriginalName" value="true" />
    </bean>

    <bean id="csvReader" class="ch.brix.genericMappingTask.lib.util.CsvReader">
        <property name="delimiter" value=";" />
        <property name="quoteChar" value="&quot;" />
        <property name="lineSeparator" value="\n" />
        <property name="ignoreEmptyLines" value="true" />
        <property name="allowMissingColumnNames" value="true" />
        <property name="encoding" value="UTF-8" />
    </bean>

    <util:list id="assetMappings" value-type="ch.brix.genericMappingTask.lib.mappingTypes.Mapping">
        <bean class="ch.brix.genericMappingTask.lib.mappingTypes.NullMapping" /><!--A: ignore, used as assetMappingFileKeyColumn-->
        <bean class="ch.brix.genericMappingTask.lib.mappingTypes.AssetTypeMapping"><!--B: AssetType-->
            <property name="valueTransformer">
                <bean class="ch.brix.genericMappingTask.lib.valueTransformer.IdMappingTransformer">
                    <property name="mappings">
                        <util:map map-class="java.util.Hashtable">
                            <entry value="10001" key="Video"/>
                            <entry value="10002" key="Image"/>
                            <entry value="10002" key="Bild"/>
                            <entry value="10003" key="Logo"/>
                            <entry value="10004" key="Icon"/>
                        </util:map>
                    </property>
                </bean>
            </property>
        </bean>
        <bean class="ch.brix.genericMappingTask.lib.mappingTypes.infofield.DateInfofieldMapping"><!--C: Creation Date -->
            <property name="infofieldId" value="155" />
        </bean>
    </util:list>
</beans>

The main player here is the GenericMappingTask-class

  • csvReader - required, reference to the CSV settings to use (see below), e.g. what separator to use.
  • assetMappings - required, reference to the mapping rules (see below). Every bean represents a column in the CSV, in that order! Most mappers also take a value transformer to transform the received value before applying it. There are a lot of different mappers, so they are not explained here in detail. We recommend that you use an IDE and import the jar as a library - this enables auto-complete of all ch.brix.genericMappingTask.lib.mappingTypes and their associated nodeFinder and valueTransformer.
  • assetMappingFile - where the CSV is located, e.g. classpath:conf/import.csv. Alternatively you can use assetMappingAssetId to refer to an asset ID inside CELUM (which is useful if you do multiple imports, so you can just add a new asset version and run the task again)
  • assetMappingFileKeyColumn - required long, in which column of the CSV the asset's path (or name) is located. This is 0-based, i.e. what your spreadsheet software would call A is 0, B is 1 etc.
  • globalMappings - optional, reference to global mapping rules, which will always be executed first, regardless of incoming data (useful for setting an asset type for example). There is also a postMappings which is run after all other mappings.
  • rootNode - required (unless in ID mode) long, from with CELUM root node to calculate the relative path from
  • ignorePath - optional boolean, ignores the assets folder path and just uses the asset name (i.e. you need to be sure they're unique)
  • pathPrefix - optional String, a path prefix that existed on disk, but is no longer present in CELUM, e.g. E:/export
  • useOriginalName - optional boolean, whether to use the asset's original file name instead of the asset name.
  • validateColumnCount - optional boolean, whether to compare the number of assetMappings with the column count of the CSV (and fail should they not match)
  • caseInsensitive - optional boolean, whether to disregard case sensitivity when constructing the path
  • removeFileExtension - optional boolean, whether to disregard the file extension

A note on shading

All mappers, value transformers and node finders are located in the generic package ch.brix.lib. These however get shaded when included in any project to _ch.brix.ARTIFACTID.lib (so ch.brix.genericMappingTask.lib in this case) in order to avoid conflicts with other plugins that may use the same libs at different version levels. When developing the mappings in your IDE, we recommend that you import the compiled jar as a library (Intellij: Project Settings > Libraries > + > Java, select the jar).

Compatibility Matrix

metadataMapper CELUM (min. version)
1.x 5.13.3

Release Notes

1.0

Released 2015-09-15

Initial release

1.x

Released every time we use it again

Updated the available mapping types (through the celum-mappings library)