Data analysis

A French writer called Jean de Lafontaine once wrote a fable about an oak and a reed: the oak makes fun of the reed for being shaken by the winds, and so frail and vulnerable. The oak feels indestructible. Then, a huge storm arrives and takes the oak down, while the reed bends, but does not break.

This story is a tale of resilience.

Resilience is a skills that I have tremendously improved in the last few months, the months of the dreaded (dramatic horror movie jingle) “data analysis”.

The bulk of my PhD data being quantitative, I ended up with massive data files from an online survey I had launched earlier this year. Although dealing with quantitative data is very different from qualitative data analysis, I think there are similarities in the processes, so hopefully everyone can relate to this story.

So, data in hand, around May-June, I figured: “How hard can it be to analyse all this? you just have to know which statistical analysis to run, learn how the software works and pretty much do it. This should be done by the end of the summer”. Such wild misconceptions on the ease of the task ahead have been common in my PhD journey. Let me tell you practically why data analysis was such a high mountain for me to climb.

First, because when you perceive a task to be hard, it is really easy to beat around the bush for ages before actually getting started. I had done data analysis in the past, but I had become highly unfamiliar with the first software and procedures I had to employ, and had to learn a whole set of new techniques and second software on top of that. Resources to help me where multiple and readily available, but I hard to pretty much teach myself to do it, which proved a long and tedious process. I tend to be stimulated by new and unknown tasks, and I love learning, but this time, it just seemed “too much”. I went slower than I normally do.

Once you get the ball rolling, you think it’s going to go smoothly. You have a fair grip of the statistics you need to run and how the software work. Things are in control. And then, you hit a wall: the data is not performing as you intended. The stats are wrong. It’s a disaster. You go back, redo everything twice, tweak your approach, iterate, modify…and most importantly, stare at your screen for hours, binge eat, freak out, lash out at your friends, and on the most glorious days, wake up at night thinking about it.

Along this mind-wrecking journey, which is still in progress, I have learned several things:

Make decisions: it’s fine to try things out, iterate, try and find the very best solution or approach, but at some point, you just need to stop, focus, and make a decision.

Seek advice: don’t remain stuck on a problem you cannot solve. Asking a quick question to someone and getting feedback will put you back on tracks.

Keep going…: even if today is not going to be the most productive, stay engaged with your analysis. Don’t leave your data hanging for a week or two without touching it, you will forget where you were and have to start again.

…but take breaks: for sheer sanity purposes. Taking a short break from your data will make you come back to it in a better state of mind and with a fresh pair of eyes.

Then one day, you’ll get past the hump and be done with it.

In the storm of data analysis, be like the reed: bend, do not break!

And you, what did you learn with your data analysis?