Support color maps for enumerated fields

[stevage]

We currently don't have any nice way to set specific colours for different enumerated values in CSV files. It would probably be reasonable to do something like:

{ "colorMap": [
  { "color": "red",
    "value": "fish"
  }, {
    "color":"blue",
    "value": "dog"
  }
]
}

C.f. offset instead of value for numeric fields.

[stevage]

Come to think of it, we should allow this for numeric fields too. Currently our colorMaps map to scale of 0 to 1, dependent on whatever the actual values end up being. In some cases it would be better to be able to do:

{ "colorMap": [
  { "color": "red",
    "value": "0"
  }, {
    "color":"white",
    "value": "50"
  }, {
    "color":"blue",
    "value": "100"
  }
]
}

Maybe with min and max properties, too. It will be interesting trying to combine this with color bins. Perhaps it just becomes a "manual' binning method.

[tobybellwood]

How close is this to #1097?

[stevage]

They're actually quite independent.

• This: allow colours to be pinned to specific values
• Per-column CSV table styling #1097: allow each column in a table to have a defined style.

[RacingTadpole]

Is this covered by #1551?

[stevage]

No. That one only covers boundaries of numeric fields. This one is about enumerated fields.

Possibly they could be unified in this somewhat awkward way:

colorBins: 3 // 3 bins, boundaries determined automatically

colorBins: [20, 40] // 3 bins, boundaries manually specified

colorBins: { "dogs": "blue", "cats": "red" } // specific values for enumerated fields

[tephenavies]

@RacingTadpole, @stevage Are you both happy if I go ahead and implement this colorBins object syntax? The only problem I see is that the other 2 versions don't specify colours, they are specified separately, so possibly it should be done like that? It looks like it could be a bit awkward in LegendHelper.js if we put the colours in colorBins.

How does setting a binning method interact with explicit colour bins?

[stevage]

Hmm I actually think I like my original syntax better:

{ "colorMap": [
  { "color": "red",
    "value": "fish"
  }, {
    "color":"blue",
    "value": "dog"
  }
]
}

Any complications with that?

[tephenavies]

I'll go ahead with the "colorMap" list of objects syntax.

[RacingTadpole]

sounds good to me!

[tephenavies]

I've come back to this issue today, and I think that it would be better to fully separate colouring of scalar and enum columns.

Proposed changes:

• colorMap and colorBins and method only apply to scalar valued columns
• new property colorCategories that can be (like colorBins) either an integer that tells Terria how many different colour stops to use, or a list of objects that fully describes

This will clean LegendHelper.js and all of the weirdness that comes from trying to treat enums like scalar valued columns.

Potential problems:

• The "cycle" method. This will presumably still need to be supported for enums (and only for enums? Doesn't seem much use for scalar values)

[RacingTadpole]

That sounds good to me - anything that cleans up the code is welcome. We should keep an eye on backwards compatibility though - eg. if you go with your plan above, add a deprecation warning if colorMap etc is used on an enum column, and just re-use colorBins instead of colorCategories for enums?

[tephenavies]

I've started making changes to add this feature and clean up the mix of scalar and enum code in LegendHelper.js, so far I have @e24b348a4. The way I'm heading, it looks like I'm going to be removing all indicesIntoUniqueValues related properties and just doing all colouring and legend creation from the uniqueValues themselves. That seems to make sense to me, do you know of any reason to retain all of these indices? The indices are another layer of indirection, are they're making O(n) operations into O(n^2), so removing them ought to speed things up as well.

Note: that commit is very WIP. There's no error checking, it works by converting values -> indices -> values just to test the concept of using values instead of indices and doesn't include the "cycle" method. (The "cycle" method will need a full uniqueValues.length lookup table if indices are removed)

[RacingTadpole]

Without looking into it in depth, that all sounds good to me. Just check where indicesIntoUniqueValues is used in terriajs, I'd be surprised if anyone else uses it. But add a deprecation warning anyway if you can. Maybe lazy calc it in the case?

Note Cycle is the default in many cases.

[tephenavies]

Where is cycle used? Is it used with columns that have a large number of unique values?

[tephenavies]

Even with "cycle", the storage and processing required is decreased from the current method, as the indicesIntoUniqueValues array takes O(m) space and O(nm) time to build (where n = values.length and m = uniqueValues.length), and the Object lookup only takes O(m) space and O(m) time.

Syntax-wise, I've chosen:

"colorBins": [
  { "color": "red",
    "value": "fish"
  }, {
    "color":"blue",
    "value": "dog"
  },
  { "color": "green" } // Colour used for "others"
]

If values are specified more than once, the last is used. Any values that are not specified form part of the "other" group.

[tephenavies]

Deprecating indicesIntoUniqueValues, indicesOrValues, indicesOrNumericalValues & usesIndicesIntoUniqueValues in TableColumn.

[tephenavies]

Does explicitly colouring "cycle" enum columns make any sense?

So, say I have a column whose uniqueValues are ['A', 'B', ..., 'Z']. I could use an explicit enum colouring:

"colorBins": [
  { "color": "red",
    "value": "A"
  }, {
    "color":"blue",
    "value": "D"
  }
]

and then any A regions/points would be red, and similar for D, then just assign ['B', 'C', 'E', ..., 'Z'] alternately to red and blue.

Is it advantageous to have this explicit colouring for cycled enum columns?

[stevage]

Hmm, there are always so many edge cases!

A very related question is (or a different view of the same question): if explicit colors are provided, how do we color any values that aren't explicitly defined?

There are three obvious possibilities:

• cycle the values provided
• color all the rest some boring color
• color all the rest according to some default scheme

I'm not sure it really makes sense to cycle values when a user has explicitly mapped colors to values - as opposed to, say, just providing a palette (colorMap). If you want Liberals to be blue and Labor to be red, do you really want blue or red to be used for any other parties?