Dissertation Data: Primary or Secondary?

Mathieu Despard

Dissertation Data: Primary or Secondary?

by Dr. Mathieu Despard

Most PhD students use data for their dissertation, so whether to use primary or secondary data is a very important decision. Primary data is data you collect, such as through surveys, scales, or interviews. Secondary data is data that has already been collected by other researchers and that you analyze.

Ultimately, the decision about using primary or secondary data depends on your research questions or hypotheses. Which type of data will be better for answering these questions or testing your hypotheses? Another consideration is whether you plan to use quantitative, qualitative, or mixed methods. Secondary data is usually analyzed with quantitative methods. Qualitative methods can also be used, though there are important challenges to address.

Here's the basic trade-off: primary data takes more time and effort to collect, but you have more of a chance to align your measures with your research questions or hypotheses. Secondary data has already been collected, yet the data may have too many limitations to be useful.

I've explained these pros and cons in more detail below (not an exhaustive list!), followed by a resource list. After you consider pros and cons, it's important to have a conversation with your dissertation committee chairperson to discuss your strategy in greater detail and consider other pros and cons not discussed here.

Primary Data

Pros

Customizing and aligning your measures. When you collect primary data, you can decide exactly what you want to measure to answer your research questions or test your hypotheses. For qualitative methods, you can design interview or focus group guides that are unique for your topic and research participants. For quantitative methods, you choose the right set of independent and dependent variables for your research questions or hypotheses. There are plenty of standardized instruments to help measure certain constructs – which is easier and less risky than designing your own instrument(s) (see below).

Social work practice impact. If you collect primary data, there's a good chance you'll be working with a research partner – an agency or organization through which you will recruit research participants, conduct data collection, and possibly design and test an intervention. This gives you a better opportunity to use findings to improve practice than with secondary data analysis – especially if you test an agency's intervention.

Novel contribution. Because you can customize your measures, you have a better chance of conducting unique and novel research, which may give you a competitive edge on the national academic job market and help define a program of research for which you will seek funding.

Online sampling and data collection help. To collect primary data, you don't necessarily need to recruit your own sample and collect your own data. There are many tools for this including Qualtrics, Amazon mTurk, and Google Surveys.

Cons

Time. If you collect primary data, chances are you will need a research partner – an agency or organization through which you will recruit research participants, conduct data collection, and possibly design and test an intervention. Chances are, you will be conducting human subjects research and need to seek and secure institutional review board approval. All this takes much more time than analyzing secondary data.

Sampling problems. If you need to recruit your own sample, it may be too small for using inferential statistics or is not representative of the population you want to better understand (external validity). You may experience study attrition, which is a problem if you want to collect data in more than one wave (e.g., pre and posttest). However, sample size is less of a challenge for qualitative studies.

Intervention problems. If you collect primary data because you want to study an intervention, there are many things that can go wrong: funding for the program dries up, agency staff leave, a change in agency policies and procedures disrupt the intervention, etc.

Designing new measures that don't work well. If you choose to design your own measures, this is a "high risk, high reward" proposition. The process can be very time consuming: extensive literature review, cognitive testing and/or focus groups, pretesting, pilot testing, and testing for reliability and validity. You might make an important scholarly contribution by developing a new measure, but you run the risk of designing an instrument that is not sufficiently reliable or valid.

Secondary Data

Pros

Time. The data have already been collected so you can focus on data analysis. If you are working with data that have been fully de-identified, chances are pretty good that your study will be determined exempt and non-human participant research by IRB.

Availability. There's quite a lot of data that have already been collected. In fact, this is an issue of growing concern among some researchers – how data that is "just lying around" is going unused. This includes data available from research consortia like ICPSR, which has data for nearly 15,000 studies, administrative data from agencies and government sources, and a variety of public available data sets. For example, I've analyzed data from the National Financial Capability Survey and the Survey of Household Economics and Decision-making.

Statistical methods. With a large sample and robust set of variables, you have more opportunities to use multivariate and advanced (e.g., hierarchical linear modeling) statistical methods if this is important for your academic goals.

More studies post-dissertation. If you find a large dataset that works well for your dissertation and it is data that continues to be collected in the field, you'll have an opportunity to produce several more studies from the same data. For example, many social work researchers have conducted studies using data from the Fragile Families and Child Wellbeing Study.

Cons

Data limitations. This is the biggest drawback. Someone else designed the instrument and defined the variables, which may not align well with your research questions or hypotheses. For example, let's say the main dependent variable I'm going to use for my research questions or hypotheses is social anxiety. I found a secondary data set, but either it does not measure social anxiety specifically and/or the measures are flawed (e.g., not established as reliable and valid or haven't been used with certain groups of people, there's a newer and better measure in the field). Thus, the central question is, can you still answer your research questions or hypotheses given the limitations of the data?

Nothing new here. With publicly available secondary data, you must be very careful not to conduct a study that's already been published. The ICPSR site tells you what studies have already been published with a certain data source, but this isn't true for all secondary data sources. For large, well known studies like Fragile Families, it can be very challenging to identify a unique study, which is fundamentally the purpose for your dissertation.

Qualitative limitations. Using secondary data for qualitative research isn't very common. It may be hard to find a secondary data source to analyze.

Related sources:

  • Boston University (publicly available datasets for social work research)
  • Federal government data (publicly available data from numerous agencies)
  • Inter-university Consortium for Political and Social Research (ICPSR) houses data from nearly 15,000 studies.
  • Measurement Instrument Database for the Social Sciences (measurement tools and instruments)
  • University of Washington (measurement tools and instruments)
    Research Methods Knowledge Base (measurement methods)

  • Funding opportunities

    by Mat Despard

    Figuring out how to finance your doctoral
    studies is an important process that warrants careful consideration of all
    available options. Continuing to work while you pursue your PhD is an excellent
    option if you can balance work and school. You keep a steady source of income
    and benefits, there may be opportunities to conduct your dissertation research
    (especially intervention studies), and your employer may offer a tuition
    benefit (true for UNC system employees). I’ve outlined additional funding
    options below, some of which apply even if you are working while completing
    your degree.

    Scholarships,
    Fellowships, and Grants

    Scholarships are grants
    awarded to students to support their studies – no repayment obligation and usually
    with no strings attached. The only requirement is to be in good academic
    standing, though eligibility can be restricted to things like being enrolled
    full-time, focusing on a certain research topic, or being a member of a group
    under-represented in academia.

    Fellowships are very similar
    to scholarships, though they often come with a research and/or teaching service
    commitment. For example, the Eileen Blackey
    Doctoral Fellowship

    supports dissertation research projects related to welfare policy and practice.
    Recipients are required to give a poster presentation at the NASW National
    Conference and submit their final dissertation. Fellowships can also be tied to
    certain practice and academic workforce goals. For example, the CSWE Minority
    Fellowship Program

    is focused on supporting researchers dedicated to culturally competent mental
    health and substance abuse services.

    Dissertation
    Grants

    are grants to help
    students complete their dissertations
    (also referred to as  dissertation fellowships) by supporting data
    collection, travel, software, tuition, and even living expenses.

    Travel
    Grants

    shouldn’t be overlooked. The UNCG School of Health and Human Sciences offers a travel grant program. These grants cover
    research-related travel, including to academic conferences. Conferences are important
    opportunities to present papers and posters, meet faculty and PhD students at
    other universities to grow your scholarly network, and learn about research in
    your field. Conference travel grants may be limited to PhD students who will be
    presenting at the conference, which means you need to submit your paper
    abstract several months in advance. For example, to be awarded a SSWR travel grant, you
    need to submit a paper abstract in April for the annual conference that takes
    place in January.

    Awards are simply
    financial gifts to recognize exemplary teaching or research. For example, UNCG
    offers an Outstanding
    Dissertation Award
    .
    External organizations may also give awards, typically to recognize
    contributions consistent with their missions.

    Assistantships

    JPhD students can serve as Graduate Research
    Assistants (GRAs) to assist faculty with their research projects. This depends
    on the availability of funding. Faculty can apply for a Graduate
    Research Assistantship (GRA) award
    from the School of
    Health and Human Sciences so they can hire a JPhD student and/or they can use
    funds from an existing research grant from a foundation, NIH, etc. to hire you.
    Thus, it’s important to reach out to faculty to learn about their current
    research projects and needs for GRAs.

    JPhD students might also serve as a Teaching
    Associate or Instructional Assistant
    to teach or help teach
    an undergraduate course or as a general Graduate Assistant with a variety of
    work tasks. In addition to GRA opportunities, students should search UNCG
    and NCAT
    job listings to find teaching associate or assistantship positions or other
    part time job opportunities outside of social work but related to your skills
    and interests.

    Assistantships at UNCG are limited to full-time
    students who have at least a 3.0 GPA and are making satisfactory progress in
    their program.

    Tips

    Identifying
    funding

    • Starting in Fall 2020, talk to the
      Associate Chair for Research, Dr. Tanya Coakley, who can help match you with
      faculty who have applied for small internal grants to fund students and/or who
      can apply for a
      GRA award
      .
    • Use online databases – start with the JPhD funding page.
    • Sign up for text alerts for various
      scholarship and fellowship opportunities.
    • Review faculty CVs which are publicly
      available on university websites and will list grants, awards, scholarships,
      etc. Identify external awards and ignore internal ones as those only apply to
      the institution where the individual earned their PhD. 

    Applying
    for funding

    • Know the deadlines and start the
      application several weeks in advance.
    • Ask faculty members who can attest to the
      quality of your work for letters of recommendation. Give them a copy of your
      updated CV, information about the award, and alert them to specific
      instructions, such as commenting on research or teaching abilities. Most
      importantly, ask faculty members no less than 4 weeks prior to the deadline.
    • If the application requires an essay, ask
      your advisor to provide feedback on a draft. Don’t get discouraged if it gets
      marked up quite a bit.
    • If you are applying for a dissertation or
      research grant, work closely with your faculty advisor or dissertation chair to
      develop research questions, methods, etc. several weeks in advance of the
      deadline (unless your dissertation proposal has been approved and you are just
      looking for funding).

    Securing
    a Research Assistantship

    • Talk to faculty about their research
      projects and need for a research assistant. This may prompt a faculty member to
      apply for university funding to support a position. This is better than waiting
      to see if a position gets listed.
    • Talk to your advisor and other faculty
      members about research teams at other universities related to your interests. These
      other universities may have research funding that can support a GRA through a
      sub-contract.

    Pursuing a PhD in Social Work: make it count

    by Dr. Mathieu Despard

    I have only one hope for anyone pursuing a PhD in Social Work: make it count. The number of people in the U.S. who have a doctoral degree has doubled since 2000, yet this represents less than 2% of the adult population. The reason is simple: getting a PhD is a major commitment of time, effort, and often, money. 

    What I mean by make it count is do something with your PhD that you couldn’t do with your MSW whether or not the position itself requires a PhD.

    What can you do with a PhD in Social Work? I’ve worked in tenure-track and non-tenure track, teaching vs. research focused, staff and faculty roles. I have several colleagues with PhDs in professional roles outside of academia.  Informed by these experiences and connections, here are some ideas and considerations. My goal here is to encourage you to think broadly and creatively about what you can do with your PhD in Social Work. Good luck! 

    Inside Academia

    Working in the academy comes with distinct advantages – a great deal of scheduling flexibility, autonomy, and yes, time in the summer to unwind or write a book or big research proposal!

    There are 4 key considerations for academic roles:

    1. Faculty or staff?
    2. Research or teaching focused faculty role, or a combination?
    3. Tenure track or non-tenure track faculty?
    4. Type of institution: R1, R2, or other? Public or private? 

    Faculty or staff?

    Most PhD graduates will naturally want to consider a faculty position to focus on teaching and/or research (see below). Yet colleges and universities also employ staff with PhDs in roles such as research associates and administrators of research centers, and in various leadership roles related to student affairs, advising, continuing education, and community programs. When you examine a college or university’s job listings, include both faculty and staff classifications. Pay close attention, however, to whether the position is hard (e.g., part of an academic unit budget) or soft (e.g., part of a time limited grant or contract) funded. 

    Research or teaching focused faculty role, or a combination?

    How do you want to spend most of your day as a faculty member? Preparing lesson plans and grading or collecting and analyzing data and writing manuscript drafts? Have you taught before? Do you find it exhausting or energizing? I spent 9 years in a teaching-focused role, teaching 8 or 9 courses per year (including summers) and was highly motivated to earn my PhD so I could conduct research and teach less. I still teach and enjoy it greatly, but I am heavily involved in research, which I also enjoy. 

    Tenure track or non-tenure track?

    Tenure track = more pressure but greater (potential) stability. The degree of pressure, particularly to publish, depends on the type of institution. Whatever the requirements to secure tenure, it’s hard work. If you finish your PhD and are looking forward to working less, a tenure track faculty position is probably not for you. Generally speaking, anticipate that the amount of effort you had to expend getting your PhD will remain unchanged until you get tenure. However, the upside is that many colleges and universities “protect” your time by giving you reduced teaching and service responsibilities during your first couple of years so you can focus on getting your research off the ground (publishing dissertation papers, launching projects). You may also receive discretionary funds to support your research!

    Many faculty members not on a tenure track enjoy stability and feel every bit a part of the faculty as tenure track faculty. In 9 years in this role, most years I worked under a single year contract, but I never felt my job was in jeopardy. Still, if a college or university runs into financial problems, tenure track jobs are better protected. And at some colleges and universities, non-tenure track faculty feel very under-appreciated and have little power. That wasn’t the case for me, but it’s a possibility. Also, non-tenure track faculty are expected to do more teaching and advising and have less time for research than tenure track faculty. But you don’t have the pressure to publish that tenure track faculty do.

    Another type of non-tenure track faculty position is adjunct. This means you teach just 1 or 2 courses per year or semester. This could be a good option if you really enjoy teaching but want to maintain a full-time social work position as an administrator, therapist, organizer, etc. In other words, it’s better to think of being an adjunct faculty member as a sort of side gig, not your main livelihood.

    Type of Institution

    The Carnegie Commission on Higher Education classifies colleges and universities based on their level of research productivity. R1 universities such as UNC-Chapel Hill and NC State are considered “very high research activity”. R2 universities such as UNC-Greensboro and N.C. A & T are considered “high research activity”. R1s place greater emphasis and resources on research and have higher research expectations (# of publications, high impact journals, grants secured) for tenure-track faculty than R2s, though pay and academic prestige tends to be higher in R1s. Teaching loads (the # of courses faculty are required to teach) may be higher at R2s, but not dramatically so. Teaching-focused colleges and universities, such as Western Carolina University and Bennett College, will have, not surprisingly, higher teaching loads – perhaps 4 courses a semester compared to 2 or 3 at R1s and R2s . 

    Public colleges and universities are larger and tend not to pay faculty as well as private ones due to lower tuition revenue and less in endowed assets. But this is probably true just among R1s and R2s; teaching-intensive private institutions may pay no more and or even less than public ones. Public institutions, which are subsidized by taxpayers (though state funding has been declining for several years now), also tend to be more public service oriented. However, the distinctions between public and private appear to be fading due to changes in state funding, tuition increases, and private institutions incorporating more public service oriented missions.

    Whether R1, R2, or teaching-focused, public or private, another consideration is whether you want to teach undergraduate or graduate students (or both). In North Carolina, there are 23 BSW and 12 MSW programs.

    Outside Academia

    There are many roles to play with a PhD in Social Work outside of academia – too many to list here, but I’ll offer a few examples. Some of these roles require a PhD, many do not. For positions that do not require a PhD, Jennifer Polk and L. Maren Wood in an article in Inside Higher Ed offer the following advice: “To build a successful, meaningful career beyond the professoriate, every Ph.D. must learn how to leverage their own distinct combination of knowledge, skills and abilities”. Put differently, think of how a PhD can help you build upon, not replace, your skills and experience.

    Advocacy Groups: statewide nonprofit organizations focus on changing systems and public policies affecting populations helped by social workers, such as NC Child and the NC Justice Center. At the national level, there are many advocacy organizations, based primarily in Washington, DC such as Prosperity Now, National Coalition for the Homeless, and the headquarters of the National Alliance on Mental Illness (NAMI). Roles with advocacy groups for someone with a PhD in Social Work include policy analyst and research director.

    Corporations: surprised to see this? Consider the recent trend of companies hiring Chief Diversity Officers, large managed care companies like Centene that need experts in mental health, chronic disease, and under-served populations, large behavioral health providers like Magellan Healthcare, and jobs in Corporate Social Responsibility and Public Relations.  

    Government: admittedly, I know the least about public sector jobs for PhD social workers. Yet state and especially federal government agencies employ PhD social workers in research, program evaluation, technical assistance, quality assurance, and administrative roles. Like foundations, public agencies benefit from staff who have a combination of relevant substantive knowledge about issues like child maltreatment and domestic violence and research and program evaluation skills.

    Philanthropy: foundations like Z. Smith Reynolds in North Carolina or foundations with a national presence like Kresge and W. K. Kellogg fund programs and projects well-aligned with social work practice and need program officers with expertise regarding issues such as mental health and homelessness, and relevant research knowledge and skills.

    Professional Associations: state and national organizations also exist to promote the interests of social workers and other related professions: Council on Social Work Education, National Association for Social Workers, National Association for the Education of Young Children. Staff with PhDs in Social Work might work in roles providing technical assistance, developing professional standards, and/or advocating for the profession with policy makers.

    Research Firms: these are organizations that conduct applied research on a wide range of topics related to social work, usually through federal government contracts and large foundation grants. North Carolina firms include RTI International and FHI360. Outside of the state, examples include RAND Corporation, Abt Associates, and Westat. These firms emphasize offering objective analysis and perspectives and do not engage in advocacy. Roles include project manager (PhD not required), and roles requiring a PhD such as Research Associate and Statistician, for which quantitative and research design skills are very important.  

    Think Tanks: think tanks are organizations that conduct applied research and policy analysis to inform public policies. Those that focus on issues of concern to social workers include the Urban Institute, New America, Center for American Progress, and MDC in Durham, NC. They bear some resemblance to Advocacy Groups in focusing on policy, yet tend not to have explicit advocacy agendas. They, like MDC, may also act in advisory roles for local and state governments.