Recent Posts

1. Welcome Abel Francis

   
   

   Abel Francis was born in the afternoon of April 22nd, 2019, at nearly nine pounds!

   Mom and baby are happy and healthy, here's the little man shortly after birth:

   

   Olson loves his little brother, seen here at a little over a month old:

   

2. Boston Marathon 2019

   
   

   What a day! Congrats to all the finishers. I count myself very grateful to be at the start line of this one, with my wife at 39 weeks pregnant. Didn't have the run that I had hoped out there, wrote some checks that my legs couldn't cash with a first …

3. Scoring arbitrarily large datasets with Pandas + Sklearn

   
   

   The workhorses of data analysis and modeling in the Python universe are undoubtedly Pandas and Sklearn. I won't extoll their virtues here, but focus on solving one limiting problem. One of the major limitations of these libraries is the size of data they can handle.

   In Pandas, the rule of …

4. Writing LaTeX in Atom

   
   

   Atom is a code editor. The defaults try to "complete" words from your writing, and don't highlight spelling. After many months of using a code editor to write, it's clear that I've gone backwards and I should at least be using spell checking!

   These two settings vastly improve the latex …

5. Tricks for coercing Pandas into parquet

   
   

   For coercing pandas date times (stored as numpy datetime):

   for col in df.columns[df.dtypes == np.dtype('<M8[ns]')]:
       # https://stackoverflow.com/questions/32827169/python-reduce-precision-pandas-timestamp-dataframe
       # apply(lambda x: x.replace(microsecond=0))
       df[col] = df[col].values.astype('datetime64[s]')
   

   For coercing python datetime (here, a datetime.date, there …

6. Some notes about running D3 inside Jupyter

   
   

   Many visualization packages rely on using D3 in the browser, and those include: Plotly, Vega, and mpld3 (links point to the code for how these projects get JS interacting with Jupyter, using IPython’s display module). Some people have no idea why it’s so hard, and I’ll count …

7. How to groupby in Pandas with a missing group

   
   

   This is a note intended for my future self. Here’s how to do it:

   1. Have a list of the values that you expect for each group.
   2. Iterate over that list, and look up the values using .loc.

   Say I want to group by months, but not all of the …

8. Boston 2018

   
   

   This was an incredible experience.

   The crowds were just amazing, and seeing my son Olson at mile 6 in the rain and cold was amazing. With the weather, Olson on my mind, and the huge crowds, parts of the race were very emotional.

   At about mile 11-13, I sped up …

9. Run rabbit run

   
   

   A brief moment of glory:

   

10. Sentiment analysis methods for understanding large-scale texts: a case for using continuum-scored words and word shift graphs

    
    

    As a grad student trying to understand the emotional content of some unreadably large collection of texts, a typical night in grad school can often go something like this: You’re up late at night planning a new research study, thinking about trying some of this fancy sentiment-based text analysis …

Page 1 / 28 »

Tweets by @andyreagan