how to deal with messy data

hint: don't

And the second thing I keep hearing people say is, my data is not in the right shape. Guys, nobody's data is in right shape. It will never be.”

Somya Kapoor of The Loops on CXChronicles Podcast (15:40)

Somya goes on to say that business teams don’t need to be analysts or even hire analysts. The trick is to use AI.

I’ve tried ChatGPT to clean up some messy data. Sometimes it works wonders. Other times I end up asking it to recognize patterns first and then give me a Python script to run some analysis. Either way, it’s allowed me to do analysis that would have been someone else’s job and to get it done much sooner than later.

Is there a moral pickle here about AI doing someone else’s job? Maybe a little. I’m not talking real data scientist work here. There’s still plenty of room (and complex data) for real analysts.

The point worth taking away from Somya is that data is messy. It will never been squeaky clean.

We hear the same thing at Accoil from a lot of companies. “We need to clean up our data before using a tool like that.” This is becoming less and less of a concern as AI can be used on the front and back end of the work, making sense of otherwise “messy” data.

There are fewer and fewer reasons to sit on your data and not learn from it. The companies embracing speed and insights over tidy tables will learn faster and grow faster as a result.

Peter

(269 / 500)