Random Test Dataset With More Values Of One Type

1 view (last 30 days)
Harel Harel Shattenstein
Harel Harel Shattenstein on 22 May 2019
Answered: the cyclist on 22 May 2019
I have a set with y values of 0 or 1.
most of my data has 0 value, so when splitting the data to train and test 80 -20 respectively, most of the test data has 0 y value.
I want that the randomly chosen test data will have more data with 1 has y vlaues, how can it be done?

Answers (1)

the cyclist
the cyclist on 22 May 2019
One simple approach would be:
  1. split your dataset into the y=0 set and the y = 1 set
  2. do the 80/20 training/test split on the y=0 and y=1 separately
  3. combine the two training sets and the two test sets
Bear in mind that highly imbalanced datasets like this have pitfalls for analysis. It's too much to describe here, but if you search keywords like imbalanced dataset machine learning, you'll be able to read about the problems, and some potential solutions.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by