Feature Transformation for Domain Adaptation: Modifying Abnormal Data to Match Normal Feature Distributions

Question

Let $X$ be a dataset consisting of $N$ instances, where each instance is described by a set of features $\text{feat}_0, \ldots, \text{feat}_m$, and let $Y$ denote the corresponding target values. Suppose that $X$ is partitioned into two subsets: $X_0$, representing normal cases, and $X_1$, representing abnormal cases, with their respective target sets $Y_0 = \{0\}$ and $Y_1 = \{1\}$.

The objective is to explore potential modifications to the feature values of instances in $X_1$ such that, after transformation, the target values of the modified instances are mapped to the target set $Y_0 = \{0\}$. In other words, the transformation should adjust the features of $X_1$ while ensuring that the modified instances exhibit the same characteristics as those in the normal subset $X_0$, effectively restoring their normality.

What algorithms or techniques are available for achieving such a feature transformation, particularly in scenarios where the goal is to align feature distributions or facilitate domain adaptation between the subsets $X_0$ and $X_1$?

Answer 1

There're perhaps many ways to achieve your specified feature transformation from the abnormal data set to align with the normal data set in terms of distribution, such as domain adaptation techniques (DAL, CycleGAN), distribution matching (Optimal Transport, MMD), and feature transformation (CVAE), etc.

A popular and practical method is perhaps the domain adversarial learning (DAL). DAL works by introducing a domain classifier that attempts to distinguish between features from $X_0$ and $X_1$, while the feature extractor ANN learns to generate features that are indistinguishable from the normal domain $X_0$. This can lead to the transformation of $X_1$ such that they appear similar to $X_0$. A variant of this is using CycleGAN which are particularly flexible even when paired data isn't available.

to train a pix2pix model to turn a summer scenery photo to winter scenery photo and back, the dataset must contain pairs of the same place in summer and winter, shot at the same angle; cycleGAN would only need a set of summer scenery photos, and an unrelated set of winter scenery photos.

Another practical method is conditional VAE (CVAE) which inserts label information in the latent space to force a deterministic constrained representation of the learned data. Therefore A CVAE can be trained with $X_1$ as the input and $Y_0$ as the conditioning label input for the encoder, and its decoder attempts to reconstruct an output that is similar to $X_0$. In this way it learns a probabilistic mapping from the abnormal feature space to the normal feature space by using the decoder to generate transformed features that should be closer to $X_0$.