Summary

fastai has been asking fastpages users to migrate to Quarto for a while now, and this blog was still running the old Jekyll-based fastpages setup from 2020. I migrated it: same content, same posts, same URLs (via redirects), new engine underneath.

What changed

The mechanical part followed fastai’s own migration guide:

• Installed Quarto and scaffolded a fresh website:blog project
• Copied the six existing notebooks and one hand-written markdown post into posts/
• Ran nbdev-migrate, which automatically rewrote each post’s frontmatter into Quarto’s format, added aliases so the old fastpages URLs (like /jupyter/covid-19/altair/2020/04/02/covid-19-dataset) keep resolving, and converted the old Jekyll YouTube shortcodes into Quarto’s native video-embed shortcode
• Restored the real site title, description, and bio from the old _config.yml and _pages/about.md, rather than leaving Quarto’s scaffold placeholders in place
• Fixed a handful of things the automated tool doesn’t touch: a couple of draft/search flags that came through as quoted strings instead of real YAML booleans, and five leftover {site.baseurl} Jekyll tags in the very first post that had to be pointed at the right relative image paths by hand
• Removed the old Jekyll scaffolding entirely — _notebooks, _posts, _layouts, _includes, Gemfile, Makefile, the fastpages GitHub Actions workflows, all of it. The repo is Quarto-only now.

The one post that fought back

Every post rendered cleanly except one: the Covid-19 county map. Two separate problems, stacked:

1. The chart itself was dead weight. It had been saved back in 2020 through classic Jupyter’s RequireJS-based chart embedding, which Quarto’s site doesn’t load. The real fix wasn’t a config tweak — it was letting Quarto re-execute the notebook and generate the chart natively.
2. Re-executing it exposed two more problems. The notebook’s Minnesota county boundary data lived at a GitHub URL that no longer exists — a plain 404, six years of link rot. And the notebook’s own code used pandas and Altair APIs (DataFrame.append, selection_single) that have since been removed from both libraries.

Rather than paper over this or quietly leave the chart broken, I:

• Sourced Minnesota’s 87 counties from us-atlas, a topojson dataset that’s actually maintained, pruned it down to just the ~21KB Minnesota needs, and bundled it directly into the repo — so the post no longer depends on someone else’s GitHub repo staying alive
• Updated the handful of pandas/Altair calls to their modern equivalents (pd.concat, selection_point) without changing what the chart actually shows
• Verified it in a real browser: the map renders, the county click-to-filter interaction works, and the linked bar chart updates correctly

Introduction

It has been so long since I update this blog. Earlier this week, github warned me about the possibility of its action being disabled soon due to inactivity in this repository. So, that is enough of a nudge for me to update this site. I want to tell my own self, via this post, “I am still learning.”

As Rachel Thomas wrote, “I usually advise new bloggers that your target audience could be you-6-months-ago.” So, what have I learned since April 24, 2020?

Credit: Photo by Faye Cornish on Unsplash

Community of Learners

What I have appreciated most looking back to my-6-months-ago self is the opportunity to learn. The concept of continuous learning has always been embedded in my training as a PhD, in which I am grateful for. But learning alone can be though. So, I am thankful for the community of learners in the Twin Cities area, particularly in Deep Learning and R. I would recommend you to find those communities in your area as well!

I was also given the opportunity to present on several different machine learning topics through these two communities. It has been a wonderful learning experience to distill them “as simple as possible, but not simpler” and transpire it to others.

In sharing the topics, I often use streamlit as my UI of choice. If you are interested in learning more about it, feel free to look at both my earlier blog post and this simple TensorFlow tutorial on sentiment analysis using deep learning in my repository.

Type-hinting in Python

In the past 6 months, I have grown to embrace type hinting in my Python code writing. As someone who works with data, providing hints on the kind of input(s) and output(s) of my code really helps me with streamlining my process and my code writing. Definitely check it out and follow along this excellent tutorial if you are interested.

A blog recommendation

I highly recommend Vicki Boykis’s takes on tech news that are rooted in humanism, nuance, context, rationality, and a little fun. Her latest writing, The rise of the non-expert expert, is quite insightful in realizing the importance of the fundamentals in tech. She will be one of the keynote speaker for rstudio::global(2021). Plus, she introduced me to bardcore!

Concluding Thoughts

Lastly, there is less than two weeks before the US election. If you can, please vote!

Interlude

It has been a tough couple of weeks for me to grapple with events that unfolded about 30 minutes away from where I live. This Martin Luther King Jr. quote that I read on Chris Damian’s blog post resonated with me:

“Let me say as I’ve always said, and I will always continue to say, that riots are socially destructive and self-defeating. … But in the final analysis, a riot is the language of the unheard. And what is it that America has failed to hear? It has failed to hear that the plight of the Negro poor has worsened over the last few years. It has failed to hear that the promises of freedom and justice have not been met. And it has failed to hear that large segments of white society are more concerned about tranquility and the status quo than about justice, equality, and humanity. And so in a real sense our nation’s summers of riots are caused by our nation’s winters of delay. And as long as America postpones justice, we stand in the position of having these recurrences of violence and riots over and over again.”

Excerpt from Martin Luther King Jr.’s speech “The Other America,” 1967 at Stanford University.

Curious on the context of the quote, I ended up listening to the whole speech on YouTube:

I hope that the words of MLK Jr. provided a context on the institutional racism that pervades the world in which we live in. May we have a more just and kind society.

Introduction

I first heard about streamlit from a data scientist in the area. Supposedly, it can create an interactive web python app in a simplified manner. So, I went ahead and run through the tutorial to understand more on its capability. Since I am familiar with flask, I am going to compare it with streamlit in this introduction.

First off, compared to flask, streamlit is simpler when it comes to creating a dynamic web app. In flask, it is recommended to save the python file in .py files in root and the html file in .html files in templates folder. This is partly due to html being a verbose markup language. On the other hand, streamlit uses GitHub flavored markdown and both python and markdown co-exist in one single python file. Markdown syntax can render rich formatted content in a concise manner. Of course, combining python with markdown texts can be confusing at first. But, if one is already familiar with jupyter notebook, this format should come naturally.

Streamlit is created with fast deployment in mind. If one wants to have a perfect platform with all the bells and whistles, flask and django should be considered. However, if we want to keep things simple and get quick feedback from other teams, streamlit should be the way to go.

With that in mind, let’s get to the streamlit tutorial.

Streamlit tutorial

My streamlit tutorial is deployed on heroku. The tutorial can be loosely divided into:

1. Interactive DataFrame
   
2. Drawing charts and maps
   
3. Add interactivity with Widgets
   
4. Show progress
5. Record a screencast

Feel free to take a look at the github repository to look into the tutorial_streamlit.py Python code.

mkdir -p ~/.streamlit 

echo "[server]  
headless = true  
port = $PORT  
enableCORS = false  
" > ~/.streamlit/config.toml  

Concluding thoughts

Streamlit package is a simple way to deploy data driven app on the internet. I am excited to use this in creating a nice interactive machine learning web app in the future!

Introduction

The novel coronavirus has upended our lives in more ways than one. To be a part of the solution, I started by visualizing the number of confirmed positive cases in the state of Minnesota. The day after, I saw the opportunity to be a volunteer Python educator for Stanford University. However, there was a problem.

I was in bed at around 11ish PM on April 3rd when I started filling in the Google Form application on my phone for this Code in Place program. The due date was April 3rd. After answering a couple questions such as, Tell us about yourself (briefly, just a few sentences), I got stuck on Record a 5 min demo of yourself teaching MoonWeight (see handout for instructions). Answer this question with a link to your video.

I almost quit. But, I know that humans are reading through this form. If I were to write an explanation as to why I did not record a video and optimize all the other parts of the application to ensure my credibility, they would reach out. So, I answered, I am writing this in bed as I just learned about it tonight and will be able to send it to you if given a chance. I finally submitted all but the video at 11:31 PM.

Two days later, I was overjoyed when an email from the Code in Place team came to my inbox requesting my teaching video for the MoonWeight challenge. I recorded the teaching part and sent it back to them hoping that I would get picked as one of the section leader.

A few days later (April 7th), the welcome email came to my inbox:

Congratulations!

We reviewed close to one thousand section leader applications and we were impressed with your teaching! This is a formal invitation to join the Code in Place teaching team…

So the journey of being a volunteer section leader for Code in Place began from here on out.

Credit: Hoi L who won the Code in Place T-shirt design

Code in Place Experience

TL;DR: it was a wonderful experience.

Final Thoughts

I wrote this post to reflect on the uniqueness of this wonderful community of students and teachers. Others have done more wonderful things than writing a post. One of the students, Regina, compiled her lecture notes on her website. The team created a podcast called Humans of Code in Place. Last but not least, one of the section leader even created a Karel Helper website. These are great examples of human ingenuity during a global crisis, so please check them out!

In conclusion, thank you. Thank you to the teaching team for letting me be a part of this experience. Thank you to my students for letting me be a part of your coding journey. Thank you to my readers for reading through this post.

Be well,

Atun

Introductions

I enjoy browsing through the data provided by the Minnesota Department of Health on Covid-19 cases daily. It provides the most up-to-date number of confirmed positive cases in the state. However, to get the historical data is a little circuitous.

So, I was thrilled to learn that the New York Times published their county-by-county Covid-19 data historically. It motivated me to create an interactive map of Minnesota and their historical data on a similar fashion as shown on the covid-19 dashboards.

In summary, using a Jupyter notebook, I want to showcase fastpages’ capacity to visualize interactive map and bar graph in Altair using Covid-19 data.

Results

Code Explanations

Here are the process to create the visuals shown above. Feel free to hit the badges above in order to see the code, or directly here.

%reload_ext autoreload
%autoreload 2
import pandas as pd
import altair as alt
import requests

# data from NY-Times:
url="https://raw.githubusercontent.com/nytimes/covid-19-data/master/us-counties.csv"
df = pd.read_csv(url)
dfmn = df[df.state=="Minnesota"]
dfmn['date'] = pd.to_datetime(dfmn['date'])

# drop the null values by first finding the index of the unknown, then use list 
# comprehension to drop the values 
unknown = dfmn[dfmn.isnull().any(axis="columns")].index
dfmn = dfmn.drop([i for i in unknown], axis=0)
del unknown
# now that there is no null value:
# convert the FIPS as string in integer format to match with the json county data for Minnesota
dfmn.fips = dfmn.fips.astype(int).astype(str)

# dealing with discrepancy with the death count: 
# use: https://web.archive.org/web/*/https://www.health.state.mn.us/diseases/coronavirus/situation.html
# and see the difference between the two dates of 03/28 and 03/27
# 03-28
# dfmn[(dfmn.date=="2020-03-28")&(dfmn.deaths != 0)]
# 03-27
# dfmn[(dfmn.date=="2020-03-27")&(dfmn.deaths != 0)]
#hide
# let's add the death data into Hennepin county for 03-27:
dfmn.loc[dfmn[(dfmn.county=="Hennepin")&(dfmn.date=="2020-03-27")].index[0],'deaths'] = 2
# new 2020-03-27:
# dfmn[(dfmn.date=="2020-03-27")&(dfmn.deaths != 0)]
# 2020-03-26
# dfmn[(dfmn.date=="2020-03-26")&(dfmn.deaths != 0)]
# I added 1 death data into Hennepin county for 2020-03-26
dfmn.loc[dfmn[(dfmn.county=="Hennepin")&(dfmn.date=="2020-03-26")].index[0],'deaths'] = 1

# getting minnesota county data
url_counties = "https://raw.githubusercontent.com/deldersveld/topojson/master/countries/us-states/MN-27-minnesota-counties.json"
resp = requests.get(url_counties)
data= resp.json()
del resp
df_county = pd.json_normalize(data['objects']['cb_2015_minnesota_county_20m']['geometries'])
df_county = df_county[["properties.GEOID","properties.NAME"]]
df_county.columns = ['fips','county']
del data

# Remove state column since it is all in Minnesota:
df3mneasy = dfmn.drop('state',1)
# grabbing all the dates in the data for the for loop
uniquedate = df3mneasy.date.unique()
# an empty DataFrame for the new data
countycase_df = pd.DataFrame()
# add 0 cases and 0 deaths for counties that do not have cases
values = {'cases': 0, 'deaths': 0}

for i in uniquedate: 
    df = df3mneasy[df3mneasy.date == i] #filter for each date
    df = df_county.merge(df,'left') # we want to keep all the counties 
    df['date'] = df['date'].fillna(i) # fill the appropriate date
    df = df.fillna(value=values) # fill the zero counts
    countycase_df = countycase_df.append(df, ignore_index=True) # new DataFrame
    
# remove df3mneasy and uniquedate
del df3mneasy, uniquedate
# getting the data for the latest date
dfmnlatest = countycase_df[countycase_df.date==max(countycase_df.date)]

# getting data to see the increment of new cases per day
newcases = countycase_df.sort_values(["county","date"],ascending=(True,True))
# grabbing all the county in minnesota
uniquecounty = newcases.county.unique()
# empty_df for 
newcases_df = pd.DataFrame()

for i in uniquecounty: 
    df = newcases[newcases.county == i]
    df['newcases'] = df.cases.diff()
    df['newdeaths'] = df.deaths.diff()
    newcases_df = newcases_df.append(df, ignore_index=True)

del newcases, uniquecounty
# I drop all the NAs because that means that on the first day, there were no differences to count
newcases_df = newcases_df.dropna()

# inspired by 
# https://github.com/github/covid19-dashboard/blob/master/_notebooks/2020-03-15-us-growth-by-state-map.ipynb
# to fix the altair transformation
alt.data_transformers.disable_max_rows()

#https://github.com/altair-viz/altair/issues/1005#issuecomment-403237407
def to_altair_datetime(dt):
    return alt.DateTime(year=dt.year, month=dt.month, date=dt.day,
                        hours=dt.hour, minutes=dt.minute, seconds=dt.second,
                        milliseconds=0.001 * dt.microsecond)

# getting the maximum and minimum date
dmax = (countycase_df.date.max() + pd.DateOffset(days=3))
dmin = countycase_df.date.min()

# getting the current_date
curr_date = countycase_df.date.max().date().strftime('%Y-%m-%d')
# getting the topographic data
url_counties = "https://raw.githubusercontent.com/deldersveld/topojson/master/countries/us-states/MN-27-minnesota-counties.json"
counties = alt.topo_feature(url=url_counties, feature='cb_2015_minnesota_county_20m')

# use selector
selector = alt.selection_single(empty='none', fields=['county'], init={'county':'Hennepin'})

background = alt.Chart(counties).mark_geoshape(
    stroke='black'
).transform_lookup(
    lookup='properties.GEOID',
    from_=alt.LookupData(data=dfmnlatest,key='fips',
                         fields=['county','cases','deaths','date'])
).encode(
    color=alt.Color('cases:Q',scale=alt.Scale(scheme='greenblue')),
    tooltip=['county:N','cases:Q','deaths:Q'],
).properties(
    width=500,
    height=350,
    title=f'Total Confirmed Cases by County as of {curr_date}'
).add_selection(
    selector
).project('albersUsa')

newcases = alt.Chart(newcases_df).mark_bar().properties(
    width=500,
    height=350,
    title="New Confirmed Cases by County",
).transform_filter(
    selector
).add_selection(
    selector
).encode(
    x=alt.X('date:T', title='Date', timeUnit='yearmonthdate',
            axis=alt.Axis(format='%y/%m/%d', labelAngle=-30,titleColor='black'), 
            scale=alt.Scale(domain=[to_altair_datetime(dmin), to_altair_datetime(dmax)])),
    y=alt.Y('newcases',
             axis=alt.Axis(title='# of New Confirmed Cases',titleColor='black'),
           ),
    color=alt.Color('county',scale=alt.Scale(scheme='viridis')),
    tooltip=['county:N','date:T',
             alt.Tooltip('newcases:Q',title='# of new cases'), 
             alt.Tooltip('newdeaths:Q', title='# of new deaths')]
)

chart = alt.Chart(dfmn).mark_bar().properties(
    width=500,
    height=350,
    title="Total Confirmed Cases by County",
).add_selection(
    selector
).encode(
    x=alt.X('date:T', title='Date', timeUnit='yearmonthdate',
            axis=alt.Axis(format='%y/%m/%d', labelAngle=-30,titleColor='black'), 
            scale=alt.Scale(domain=[to_altair_datetime(dmin), to_altair_datetime(dmax)])),
    y=alt.Y('sum(cases)',
             axis=alt.Axis(title='# of Total Cases',titleColor='black'),
           ),
    color=alt.Color('county',scale=alt.Scale(scheme='viridis')),
    order=alt.Order(
        'cases'
    ),
    opacity=alt.condition(selector, alt.value(1), alt.value(0.5)),
    tooltip=['county:N','date:T','cases:Q', 'deaths:Q']
)

final_chart = alt.vconcat(
    background, 
    newcases,
    chart, 
).resolve_scale(
    shape='independent',
).configure(
    padding={'left':10, 'bottom':0}
).configure_axis(
    labelFontSize=10,
    labelPadding=10,
    titleFontSize=12,
).configure_view(
     stroke=None
)

final_chart

Concluding thoughts

I hope you enjoy the visualization and the breakdown summary on the process that I take to create this Altair visualization. The module is full of wonderful ways to create visualization that can enhance your analysis. Feel free to explore their gallery for more awesome graphs.

If you have any comments or would like to connect, add me on my LinkedIn profile!

Introduction

In the past couple of weeks, the Introduction to Machine Learning for Coders by Jeremy Howard has helped me better understand Random Forest technique and provided me with a nice introduction to deep learning.

Then, Covid-19 happened.

Since I am a fan of fast.ai, I listened to Jeremy’s YouTube summary on the situation. At the end of his video, he pointed his audience to the fast.ai forum.

When I searched through it, I stumbled upon the beautiful Covid-19 Dashboards website. The beauty and simplicity of the format is what really sold me into creating my site using this fastpages.

Setting up

In terms of setting up fastpages, it is pretty self-explanatory. I am thankful to Abdul Majed for his nice walk-through video that helped reassure me that I am on the right track.

Yet, I find myself stumbling through what would be the next step for me once I have the blog template up and running. So, here is my attempt to share with you what I have done to customize the Index Page and About Me page.

Summary

I hope that this post will help others who are trying to set up their Index Page and About Me fastpages site. Feel free to reach out to me on LinkedIn or Twitter if you have any questions!