I'm just bursting with joy over having achieved 99.39% and 98.92% of inter rater reliability (IRR) in the coding of my dissertation data. For those of you not familiar with IRR, best practice within research encourages a researcher to achieve at least 85% in IRR. With the scores listed above, I'm able to confidently say that after multiple rounds of coding among multiple coders, that the data has been consistently adequately and thoroughly coded!
This was no easy task as over the course of the last 3 months, a second coder and I have been tirelessly working on creating, redefining, collapsing, aggregating, and fine tuning the codes as we worked with the data across state lines from Pennsylvania to North Carolina. Granted, in the big picture of things, reporting these percentages will be just one sentence in my write-up of the results but goodness gracious, what work it took to get the content of this one sentence!
From endless emails and texts to sacrificing personal and professional time, to using precious paper and ink to print the transcripts to listening to the transcripts to ensure accuracy of what was typed to conducting trainings in person (even for up to three hours at a time on a Sunday afternoon while my poor parents waited for us to finish working in a coffeeshop) to making trips to the post office in order to mail packages of coding back and forth, Lauren and I pulled through and achieved an IRR score I would have never dreamt of obtaining. It truly does take a team to pull off this effort, including the help of Meredith & Ashley in learning my way through Excel, and others who have reached out to assist me with learning how to better use the year membership of a qualitative computer software my family so generously purchased as this past year's birthday present. I truly could not be doing this dissertation, and the last 3 years of the PhD program, without the support of loved ones and colleagues.
Granted, this IRR information is just one minute step in the dissertation process but gosh, it feels like I just scaled a bit of the mountain!! I can now, in good faith and in conjunction with the member checks that have been conducted with the participants in my study, move forward to analyzing the data that has been coded (which is the fun part!!).....just as soon as I hear from my committee by hopefully August 1st.
Fingers crossed!!!
This is what a coding training session looks like at the Cocoa Beanery in Hershey, PA ;) |
SO THANKFUL for Lauren giving up a portion of her end of the school year and summer to help me with the research process!! |
No comments:
Post a Comment