Monday, January 28, 2013

Astronomy vs. Data Science

In response to my last post about the transition from Astronomer to Data Scientist many readers wanted to know the pros and cons of academia versus tech. Below I outline a few of the major differences between these career paths. Obviously, there is a lot of variety in individual companies, institutions, and experiences -- so please understand that the below is simply my (somewhat biased) perspective.

According to a study by the American Institute of Physics, the median starting salary for Astronomy PhDs at academic institutions is $50,000 (although it isn't clear if this is based on 9 or 12 month contracts). Post-docs tend to make between $45,000 - $65,000 annually (on the higher end for a fellowships or national lab appointment, on the lower end for a university appointment). Astronomers (at all levels) report an average annual salary of $87,000. At top research institutions, this is of course higher -- for instance Berkeley assistant professors make an average of $110,000. Many institutions offer additional benefits like pensions, summer stipends, and regular sabbaticals. My faculty offer (at a teaching institution) was $50k/$60k for a 9/12 month contract.

Starting salaries for PhD data scientists according to Insight Data Science Fellowship are between $90,000 - $130,000 base salary, plus an average of 10% annual cash bonus, and additional annual stocks/equity bonuses (mine is ~10% of my base salary). Many tech companies also have perks like free food, transportation, gym memberships etc.

Take away: The average data scientist starting package is 2.5X that of the average astronomy post-doc, and 1.4X that of the average professional astronomer. 

I work 45-50 hours a week, I am not expected to work evenings or weekends. Hours tend to be longer at smaller companies, start-ups, or companies that are pre-acquisition/IPO. I get 15 vacation days and 12 holidays a year (this will increase to 20 vacation days after 4 years). My job is such that it is pretty easy for me to work remotely or at home, but this obviously varies from company to company. One of my teammates, who has a young daughter, works one day a week at home to be with her.

I honestly don't know how many hours post-docs and faculty work. I've heard 60-70 hours per week mentioned. I work less hours per week at my current job than I did in graduate school. However, academics get huge amounts of vacation, and many people (especially at less research-driven institutions) take these summer/winter breaks. I typically took more than three weeks annual vacation when I was in graduate school. Also, because there tends to be less required "face time" for academics, there is a lot of flexibility in when and where you work these hours. This can be especially helpful for women juggling careers and children or other family responsibilities.

Take way: Tech industry professionals work less hours than academics. Academics have more flexibility in their schedule and more vacation time.

Job Availability
As of my writing this post, there are 335 jobs posted on the astronomy rumor mill for next academic year. There are approximately 200 astronomy/astrophysics PhDs awarded annually. According to this AIP study, 75% of recent astronomy PhD grads are able to find postdoctoral positions. However, there are ~4X more postdoctoral positions than there are tenure track faculty positions. This means, that the majority of people who are currently in astronomy post-docs will not be able to continue in academia at the faculty level.

The McKinsey Global Institute estimates that in the next five years the United States alone will have a talent shortage of 140,000 to 190,000 jobs for analytics / data-science positions, as well as the need for approximately 1.5 million managers in these departments. Meaning that there will be that many positions that need filling, without people to fill them. My team has been constantly hiring/searching for over a year now.

There are several orders of magnitude more jobs in the tech industry than in astronomy. This has many consequences for the job seeker. You have much more flexibility in the location and specifics of the tech jobs you pursue. A major appeal for me in making the transition to tech was the ability to stay in the Bay Area for the foreseeable future.

Most recent PhDs pursuing academic jobs need to be willing to relocate for 5-10 years for various term/post-doc appointments. This can cause major problems for couples or people who aren't able/willing to relocate or live wherever they get a job offer. I feel confident that even if I do not stay at my current position, I will be able to find future tech jobs without having to relocate. Women specifically find the timeline of post-docs problematic because these are prime child-bearing years, yet also a time when their job tenure and location is the most unstable (and they get paid less if they are post-docs).

Take away: A benefit of a larger industry allows tech jobs seekers more flexibility in location, specialization, and work environment over academic positions. Post doctoral positions are fairly easy to obtain out of graduate school, however the majority of astronomy PhDs will not secure tenure track faculty positions.

Job Security
At a tech company you will most likely be an "at-will" employee. This means that you basically have no job security. While it would be bad practice for a company to fire people for no reason, and tech employees tend to fair well in times of recession, ultimately you can be let go anytime for any reason. People who aren't performing up to expectations are regularly asked to leave my company.

Tenured professors are one of the most secure jobs in existence. I've never heard of a post-doc that has been removed from their position before their 3/5 year contract is up. Even "soft-money" scientists at national labs tend to stay in their positions for many years. If you are able to get a job in research/academia, you will probably not get fired unless you break the law or are grossly negligent.

However, as mentioned above in academia there is much less flexibility in the location of where you will find a position, and post-doc jobs are short in tenure and usually require frequent relocation. Also most post-docs will not be able to secure tenure-track faculty positions, so there is not long-term job security for most post-doctoral scholars.

Take away: Tenure-track academic positions are much more secure than tech jobs. Tech jobs tend to be shorter in tenure. Post-doctoral scholars have much less security. Tech jobs offer more long-term stability in location right out of grad school.

Project Length
Academic projects tend to last for months to years, allowing the researcher to dig incredibly deep into the subject matter and really explore something as thoroughly as possible.

Tech projects tend to be much shorter in tenure. Because projects need to contribute to the value of the business, it is rare to be able to work on anything without quickly producing results that have business impact. My projects tend to be on the order of days, not weeks. This means that the complexity of the problems I work on is not the same as academic research. Note: this is different in research divisions of companies (like Microsoft/Google/IBM research).

Take away: Project time-lines tend to be shorter in tech than academia. This results in less complicated or in-depth projects.

Project Direction
In academia, researchers get to explore projects which interest them. They get to "be their own boss," and pursue their own paths and directions with their work. Of course there are grant committees and collaborators which influence work, but for the most part the researcher has a lot of flexibility in what they spend their time researching.

In tech there is much less flexibility. For the most part, projects are not pursued unless they will improve the company's business value or the customer experience. This means that as a data scientist the direction of my work is determined not by what interests me most, but by what will help the company most. There are opportunities to be creative (like hack days, or 20% projects) but these are not my primary duties.

Take away: People in tech jobs have less creative control of the direction of projects when compared with academics.

State of the Field
Data science is a fairly new discipline. This means that there are many problems that have been unsolved and many "low-hanging fruit" projects that are simple, but can have real impact on the tech community and world as a whole. A new data scientist has the potential to perhaps have more impact in this discipline, than a new astronomer where many of the easier/simpler problems have already been solved. That being said, with the projected potential growth of the data science field in the next five to ten years this may not be true for very long.

Take away: It may be easier to have an impact on a newer discipline like data science than a more established field like astronomy.

Subject Matter
One of the hardest things about leaving astrophysics is that I no longer get to study astrophysics. While many of my day-to-day tasks are the same -- data analysis, programming, statistics, presenting results, designing experiments -- the subject matter is not quasars and galaxies, but customer behavior. This creates a new set of interesting challenges: it turns out people don't behave as predictably as galaxies. I enjoy that the work I do is directly applicable to helping people and that I see the fruits of my labor almost immediately impact the company. I also miss studying the universe.

Take away: Tech jobs have a very different set of subject matter. Pick a company/product that you are interested in before making the transition.


Alyssa said...

Great post - thank you for sharing! I'd like to see a break down of how one spends their days in the two positions. For example, TT professors tend to spend much of their time grant writing, managing their group/lab, teaching, much of their time is actually spent doing research on topics they are interested in?

Nan said...

Great summary! I want to add something to the Job Security section.

Although post-docs rarely get fired before their contracts end, most of them leave when their contract end. In some areas like Electrical Engineering, a post-doc contract usually last 1~2 years. The cycle of applying to an academic position lasts almost one year. So post-docs need to keep looking for jobs. If you ask a postdoc in these areas, "where will you be next year?" most of them are not able to answer. So I think post-docs don't have job security.

berkeleyjess said...

Nan, thanks for the feedback. I've added a bit more discussion to the job stability / availability sections to reflect this.

Caroline Simpson said...

Academics generally get *unpaid* vacation time. Most of us have 9 month contracts; summer funding is either from grants (but NSF limits to 2 months) or summer teaching if available (not always). I literally do not have anything in my paycheck for "vacation" (just sick leave). But yes, I believe my time is more flexible in general. Except I can't take time off during semesters because of teaching.

Nick Ball said...

Excellent summary. Thanks for posting.

Unknown said...

Great post! I wonder what about pension and EI. In Canada, postdoc's salary is not pensionable or insurable, but certainly taxable.

Anonymous said...

Interesting read, thank you for all the info. Quick question: what about foreigners? They unfortunately need the post-doc/faculty jobs to secure their visas and then later green cards. Or will Yammer and other places support H1B visas and green cards?

Unknown said...

Great post! Thanks for sharing. I wonder about pension and EI. In Canada, as a postdoc, we are not qualified for pension and EI because a postdoc position is considered as an apprentice.

berkeleyjess said...

My company does not supply a pension, but we do have 401k's and company matching, I think this is pretty standard. I don't think post-docs are provided a pension by the university, in fact most universities and national labs are getting rid of pensions for their new employees.

We have hired quite a few foreigners, and my partner who is non-american also got hired by a tech company. I believe most companies will help you secure a visa and then help you apply for a green card. They just need to demonstrate that you are specialized enough that there isn't an american who could take your place, which is usually easy to do for people with PhDs.

Anonymous said...


Thank you for your answer regarding the hiring of foreigners. It may be an option to pursuit. Unfortunately, us in sciences are not educated on how to find jobs outside of our field, we just know the usual suspects: AAS jobregister and fellowships. We don't even know where to look for these good jobs and how to approach them. It should be taught, especially considering the low hiring rate as you go up the academic latter. It should be made clear in the beginning, that as a graduate student you are hired to do manual labor and in the process you will get a PhD, which you can use to get jobs at such and such places. Us in astronomy can not even imagine doing anything else.

A side note: There may the x amount of post-doc jobs and y amount of graduates, which makes the scenario seem not so bad, given the reasonable y/x ratio, but you left out the part that the majority of post-docs reapply for another post-doc or two and that significantly decreases the actual likelyhood of a fresh graduate getting a job, especially that more experience postdocs are generally more preferred and are taking the majority of prize fellowship positions.

Unknown said...

Great article, Jessica. Thanks for posting! I myself have been interested in data science and analytics for some time now, so I was particularly interested in your comparison. I also want to make a comment regarding the broader impact of articles like these.

When I started thinking seriously about the direction I wanted to take my career after grad school, one of the problems I encountered when checking out a particular field was getting a sense of what day-to-day life was like in that profession. Would I be happy? Part of the problem was that I wasn't really sure even what questions to ask. Add to this the weak support provided by the department at my university for students considering careers outside academia, and one really faces an uphill battle finding the relevant information. It's so refreshing, then, to read your perspective on the path that you've chosen in a career many of us are unfamiliar with and compare it to your life in grad school, which most of us are familiar with. That you've broken things down along aspects that really make a difference in one's quality of life is all the better. Even for people not interested in data science, it's worthwhile to consider your comparison and ask the same questions when they find a field they are interested in.

Both fields are good. All fields are good. Ultimately, the aspects you mention are enough to separate most fields and it comes down to a personal choice which you choose. All the better to make an informed choice.

berkeleyjess said...

According to the AIP study, 75% of recent astronomy PhD grads go into post-docs. This data is from 2007-2009, and the market has gotten more competitive since then with less faculty jobs and more people doing 3-4 post-docs.

My experience is that there was a lack of knowledge/support in my department regarding how to pursue jobs outside of academia. I had to be very self-sufficient in finding opportunities on my own and figuring out how to make the transition.

There are groups of people who are trying to create a community for people who are wanting to make the transition out of academia. For instance there are several groups on LinkedIn for this purpose:

Astronomers Beyond Academia
Alternative PHD Careers

Anonymous said...

Like Steven, I worry I won't like what I find if I decide to leave academia after my studies are done. Part of what I enjoy most about astronomy (aside from studying the universe) is in fact the data analysis portion of my work. I'm just not sure if that would translate into enjoyment in the data science field.

Which leads me to my first corny question that you probably get a lot: could you give a rundown of a typical day? Or, instead, examples of projects you have worked on? What type of analysis is generally involved?

And my second question is probably another typical one: how did you go about getting your foot in the door? Did you go looking through lists of job openings? Or did you contact particular companies that you were interested in? Have you worked for multiple companies? I've heard from several sources about the possible future shortage of data scientists, but it's unclear to me where these jobs are or how to find out more about them. Any advice?

Thank you so much for both this article and the previous one. There are so many graduate students who are so hungry for this type of information; it's great to finally see it getting out there into the academic world.

Unknown said...

Hey Anonymous,

I can comment a bit about getting your foot in the door, although not with the same authority as Jessica. The first thing I would suggest is leveraging the networking and newsfeed powers of LinkedIn. There are several groups that many professionals within data science use to post interesting articles and questions related to data science. Be warned that a fair amount of it is bs because of all the hype surrounding the market right now, but there's also a lot of good stuff, too. Typically, these groups are very active and responsive. Some of my favorites are Big Data and Analytics, Data Science Central, Big Data SF Bay Area, and Data Scientists. Also, check out the pages of some of the leaders in the field to see what groups they're in. Some people you can even follow. (I just started following DJ Patil, who is one of the big names and posts articles regularly.)

The second thing I suggest that you seriously consider is the Insight Data Science Fellows Program. It's a 6-week program specifically designed for post-graduates to bridge the gap into industry data science jobs. They've only done a couple sessions, but I've heard only good things from the people who've done the program and the graduates of the program get offered jobs at top companies. Also, Jake, the founder, is a really great guy full of passion and ideas.

I hope these couple points help. If you run across any tips yourself, I'm sure we'd all be glad to hear them. Feel free to connect if you have any more questions. Open invitation to anyone.

Also, don't forget that Jessica is the expert because she has a real job already and actually DID bridge the gap!

berkeleyjess said...

Dear Anonymous,
I agree with everything that Steven has said. You might want to read my first post about this topic, where I discuss suggestions of how to make the transition from academia to tech. I will also write a future post either here or on types about my typical day as a data scientist and the of projects I do.

Anonymous said...

Job security for postdocs can also depend on conventions. I was hired with an email "expectation" - but not contract - of 3 years. Then, I was told (half-way through job-application season) in the second year that I couldn't rely on the third year. This was a major source of stress and insecurity at the time, especially as it was something that would not have happened at my grad school institution so it kind of blindsided me.

Sarah said...

Thank you for this post. I am in my last semester as a graduate student and am struggling with the decision to continue on with a postdoc, or to transition into data science or other non-academic positions. I'm afraid I'll miss the science, but the advantages of data science are very strong.

Anonymous said...

Hi Jessica,
Thanks for posting your column "Astronomer to Data Scientist" on the AAS website yesterday. It reached me at a time that I am seriously considering my future in astronomy at a time that living on grant money becomes increasingly hard while waiting for JWST to revive my field of expertise.

In your postings "Astronomer to Data Scientist" and "Astronomy vs data science" I don't see the topic of age tackled. What are the chances of a 40+ year old astronomer (with 17+ years of IDL experience ;-) to get hired as a data science worker?

deepika said...

That's an great article sharing about Data Science.
Learned a lot from this article.