Colorant conversion and channel layout

Augmentor has warpped some commonly used basic operations so that you can use to build the augmentation pipeline. The internal column is what you'd probably do outside of Augmentor.

Category	internal	Augmentor
Conversion	`T.(img)`	`ConvertEltype(T)`
Information Layout	`ImageCore.channelview`	`SplitChannels`
Information Layout	`ImageCore.colorview`	`CombineChannels`
Information Layout	`Base.permutedims`	`PermuteDims`
Information Layout	`Base.reshape`	`Reshape`

It is not uncommon that machine learning frameworks require the data in a specific form and layout. For example many deep learning frameworks expect the colorchannel of the images to be encoded in the third dimension of a 4-dimensional array. Augmentor allows to convert from (and to) these different layouts using special operations that are mainly useful in the beginning or end of a augmentation pipeline.

using Augmentor
using ImageCore

# 300×400 Matrix{RGB{N0f8}, 2} => 300×400×3 Array{Float32, 3}
img = testpattern(RGB, ratio=0.5)
img_in = augment(img, SplitChannels() |> PermuteDims(2, 3, 1) |> ConvertEltype(Float32))

# 300×400×3 Array{Float32, 3} => 300×400 Matrix{RGB{N0f8}, 2}
img_out = augment(img_in, ConvertEltype(N0f8) |> PermuteDims(3, 1, 2) |> CombineChannels(RGB))

img_out == img

true

References

Augmentor.ConvertEltype — Type

ConvertEltype <: Augmentor.Operation

Description

Convert the element type of the given array/image into the given eltype. This operation is especially useful for converting color images to grayscale (or the other way around). That said, the operation is not specific to color types and can also be used for numeric arrays (e.g. with separated channels).

Note that this is an element-wise convert function. Thus it can not be used to combine or separate color channels. Use SplitChannels or CombineChannels for those purposes.

Usage

ConvertEltype(eltype)

Arguments

eltype : The eltype of the resulting array/image.

Examples

julia> using Augmentor, Colors

julia> A = rand(RGB, 10, 10) # three color channels
10×10 Array{RGB{Float64},2}:
[...]

julia> augment(A, ConvertEltype(Gray{Float32})) # convert to grayscale
10×10 Array{Gray{Float32},2}:
[...]

source

Augmentor.SplitChannels — Type

SplitChannels <: Augmentor.Operation

Description

Splits out the color channels of the given image using the function ImageCore.channelview. This will effectively create a new array dimension for the colors in the front. In contrast to ImageCore.channelview it will also result in a new dimension for gray images.

This operation is mainly useful at the end of a pipeline in combination with PermuteDims in order to prepare the image for the training algorithm, which often requires the color channels to be separate.

Usage

SplitChannels()

Examples

julia> using Augmentor

julia> img = testpattern()
300×400 Array{RGBA{N0f8},2}:
[...]

julia> augment(img, SplitChannels())
4×300×400 Array{N0f8,3}:
[...]

julia> augment(img, SplitChannels() |> PermuteDims(3,2,1))
400×300×4 Array{N0f8,3}:
[...]

source

Augmentor.CombineChannels — Type

CombineChannels <: Augmentor.Operation

Description

Combines the first dimension of a given array into a colorant of type colortype using the function ImageCore.colorview. The main difference is that a separate color channel is also expected for Gray images.

The shape of the input image has to be appropriate for the given colortype, which also means that the separated color channel has to be the first dimension of the array. See PermuteDims if that is not the case.

Usage

CombineChannels(colortype)

Arguments

colortype : The color type of the resulting image. Must be a subtype of ColorTypes.Colorant and match the color channel of the given image.

Examples

julia> using Augmentor, Colors

julia> A = rand(3, 10, 10) # three color channels
3×10×10 Array{Float64,3}:
[...]

julia> augment(A, CombineChannels(RGB))
10×10 Array{RGB{Float64},2}:
[...]

julia> B = rand(1, 10, 10) # singleton color channel
1×10×10 Array{Float64,3}:
[...]

julia> augment(B, CombineChannels(Gray))
10×10 Array{Gray{Float64},2}:
[...]

source

Augmentor.PermuteDims — Type

PermuteDims <: Augmentor.Operation

Description

Permute the dimensions of the given array with the predefined permutation perm. This operation is particularly useful if the order of the dimensions needs to be different than the default "julian" layout (described below).

Augmentor expects the given images to be in vertical-major layout for which the colors are encoded in the element type itself. Many deep learning frameworks however require their input in a different order. For example it is not untypical that separate color channels are expected to be encoded in the third dimension.

Usage

PermuteDims(perm)

PermuteDims(perm...)

Arguments

perm : The concrete dimension permutation that should be used. Has to be specified as a Vararg{Int} or as a NTuple of Int. The length of perm has to match the number of dimensions of the expected input image to that operation.

Examples

julia> using Augmentor, Colors

julia> A = rand(10, 5, 3) # width=10, height=5, and 3 color channels
10×5×3 Array{Float64,3}:
[...]

julia> img = augment(A, PermuteDims(3,2,1) |> CombineChannels(RGB))
5×10 Array{RGB{Float64},2}:
[...]

julia> img2 = testpattern()
300×400 Array{RGBA{N0f8},2}:
[...]

julia> B = augment(img2, SplitChannels() |> PermuteDims(3,2,1))
400×300×4 Array{N0f8,3}:
[...]

source

Augmentor.Reshape — Type

Reshape <: Augmentor.Operation

Description

Reinterpret the shape of the given array of numbers or colorants. This is useful for example to create singleton-dimensions that deep learning frameworks may need for colorless images, or for converting an image array to a feature vector (and vice versa).

Usage

Reshape(dims)

Reshape(dims...)

Arguments

dims : The new sizes for each dimension of the output image. Has to be specified as a Vararg{Int} or as a NTuple of Int.

See also

CombineChannels, augment

Examples

julia> using Augmentor, Colors

julia> A = rand(10,10)
10×10 Array{Float64,2}:
[...]

julia> augment(A, Reshape(10,10,1)) # add trailing singleton dimension
10×10×1 Array{Float64,3}:
[...]

source

This page was generated using DemoCards.jl and Literate.jl.