Recognizing Subjective Research

Indi Young
2 min readMar 27, 2023

--

Pop culture fails to prove or disprove anything because they reach for poor research technique

I want to help you learn to recognize something. I’ll use an example from a podcast. 🎧

The podcast host questions two researchers about their published theories on creativity. He’s a creative person themself: a writer, podcaster, and musician. The host is not sure the published theories are valid. 🤨

Both researchers are trying to help organizations innovate. They looked at data about songs and cookbooks. They correlated conditions when a songwriter (actually a band) or author had a hit but has not produced another. They came up with the explanation that one-hit-wonders are because people are afraid to ruin their hit by creating something else.

In the podcast interview, it was apparent that the researchers had not listened to many people working in a creative field to understand the breadth of reasoning and emotion involved.

👉 First Recognition - Listening to a variety of people shows how a variety of approaches are all valuable. Relying on numbers alone invites assumption.

Later in the episode, the researchers reveal that in both published papers they apply this theory to creators in all sorts of fields.

👉 Second Recognition - For some reason society loves a universal model. It can work in physics, but in social research, universal models aren’t actually universal. Worse, universal social models other all of us who are not defined therein. 😕

I was eagerly following the episode. 🎧

The host introduced two creative people: a band member and a cookbook author. Both had one-hit-wonders. Both spoke to the host about the swirl of circumstances preventing them from making a second hit. In the cookbook author’s case, the second potential hit is actually in progress. Their stories are full of interior cognition and are creatively unfolded by the host.

And here is the…

👉 Third Recognition — Disproving the researchers’ published theories with only two people (or three, if you count the host) doesn’t quite get out of the “subjective” end of qualitative research.

Empirical qualitative research is defined by patterns. To find patterns, you need a more people. You can maybe get away with 5 or 6 people if you have a very narrow frame for the study and for the people you are studying. But “being creative” is way too broad of a frame.

Two double-ended arrows, one representing quantitative research and the other representing qualitative research. Definitions of the kind of knowledge generated by each is below each arrow. Quant is defined as “quantity, amount, scale.” Qual is defined as “patterns, regularities, differences.” The key point of the diagram is that each arrow is a continuum from subjective to empirical. And for Qual, subjective tends to take the form of a couple of anecodatal stories.

It was a fun episode, but the researchers’ papers were not actually disproved.

#UXresearch #DesignResearch #BigData #FrameYourStudy

--

--

Indi Young

Qualitative data scientist, helping digital clients find opportunities to support diversity; Time to Listen — https://amzn.to/3HPlESb www.indiyoung.com