Recent Posts

  1. Sharing code

    Fri 22 July 2022

    Let’s say you have a codebase X, and a codebase Y. X produces some results or does some logic that are needed by codebase Y. What should we do to get the results (for this exercise, let’s say it’s a JSON file) from X to Y?

    • Shared …
  2. AI-powered Fitness Training

    Thu 21 July 2022

    Last year Alex Hutchinson, in his generally excellent Sweat Science series, discovered the four level framework for data analytics:

    • Descriptive: What happened?
    • Diagnostic: Why did it happen?
    • Predictive: What will happen?
    • Prescriptive: How do we make it happen?

    This isn't news for people who work in data, and yet he …

  3. 2022 Boston Marathon

    Fri 01 July 2022

    Didn't hit the mark this year, and likely went too hot for the first half. Bonus: either caught COVID during the race, or dampened the immune system enough to allow the infection that I was fighting already to go full force.

    Strava result

  4. Non-SQL in your DBT pipeline

    Thu 31 March 2022

    DBT forces you to use SQL, even when Python or R transformations are clearer.

    DBT makes a lot of sense for 80% of data pipelines, but there are steps that it just isn’t worth forcing into SQL (even when that is possible!). If the code is simpler and easier …

  5. The “Pandas Way” is the functional API

    Wed 30 March 2022

    With Pandas ≥ 1.0, the functional API is powerful and should be the new standard.

    Someone just pointed me to the pyjanitor package, which I don't actually think is very useful with Pandas >= 1.0, because the functional API for Pandas is quite powerful these days. The examples that the …

  6. Business logic in DBT

    Mon 14 March 2022

    There are many possibilities for implementing custom logic, and this framework can help you sort through the options.

    I have been really enjoying using the DBT to structure data pipelines, the framework it enforces requires you to structure your pipelines in a standardized way. The tool meets perfectly in the …

  7. Whoop Heart Rate Accuracy

    Mon 27 December 2021

    In the fall of 2020, I was curious about Whoop and hearing good things from some of the people I follow (Ted King). In the spirit of amateur fitness enthusiasm and research adjacent to MassMutual's wearables program, I ordered a 3.0 band in the October 2020.

    Overall, I have …

  8. Jenkins blocking job

    Fri 17 December 2021

    Say you have a job, that depends on three other jobs having been completed. They all run at the same time, and are not explicitly a part of your pipeline (you can't put them as a single multijob step). You need you job to depend on at least one of …

Page 1 / 7 »