Just in case you missed my previous posts in which I introduced the Notion budget manager project you can check them out here
A semi-automated budget manager with Notion: an overview
A semi-automated budget manager with Notion: building the template
Today I’m going to explain how I automated a few tasks with a little Python code and the Notion API.
Tools I used
Before we start, I think it’s useful to list here the tools and languages I used and explain why I chose them.
As I already mentioned, I used Python, specifically the version 3.9, as a programming language mainly for two reasons — the first one being that I simply happen to know the language (although I can’t call myself an expert) and the second one being that there are a few libraries written in Python that offer an easier interface to the Notion API. The one I chose for this project is notion-sdk-py by ramnes since it is simple and intuitive, works well and at the time I started the project had the most recent commits - alternatively there is also notion-py.
As a “canvas” for my code, I chose to use notebooks on Deepnote: in case you don’t know, Deepnote is a cloud service that works very similarly to Google Colab and other similar web-hosted services that allow you to write and execute Jupyter Notebooks on dedicated machines. I chose this service for 2 specific reasons: the first one is that code blocks and their relative outputs can be embedded in Notion just as iframes - this is a requirement for building the interactive dashboard - and the second one is that notebook runs can be easily scheduled for running automatically daily or weekly.
If you want to duplicate and/or modify the project, which is available here for free, you have to be registered to the platform - if you’re not you can create an account for free, and if you use this link you’ll receive an additional 20 Pro machine hours to run your projects 😉.
For the dashboard, I’ll use plotly to generate charts and just a little of HTML and CSS (nothing too complicated).
Create and manage a Notion integration
Before we even start writing code however, we need to do a very important thing, that is creating a Notion integration. To do so, follow this link https://www.notion.so/my-integrations and click on the button “+ New integration”: a new page will open, requesting a name for the integration (put here something easy that explains what the integration does, for example “budget manager automation”), an optional logo image, a workspace to associate the integration (for personal accounts there is just one so no need to worry about that) and finally the content capabilities, check all of them and finally hit “Submit”. Once the integration has been created, its internal integration token will be available - we will need this to make everything work.
Once the integration is associated with the workspace, we can go to the home page of the budget manager in Notion: here in the top right corner you can see a “Share” button - in the input select the integration you just created and now you’re good to go!
Setting environment variables
Now, back to Deepnote. You might have noticed that in the notebooks there is a section called “Initialization” where you’re asked to create an “environment variable integration” and to set the variables listed afterwards.
The variables in question are the NOTION_SECRET in which we should save the internal integration token we mentioned before, and a bunch of database ids - the ones that are present in the template. You are, of course, free to modify the code as you see fit and to add or remove variables and/or other pieces of code, however keep in mind that nothing will work if NOTION_SECRET has not been set.
To create an environment variable integration in Deepnote, simply go in the “integrations” section of the panel on the right part of the screen, click on “create new” and then choose “Environment variables”. In the “Key” field you should put the names listed in the section, while in the values field you have to put the database id.
To retrieve the database id, go back to Notion and select the page that contains the database of interest: in the top right corner click on “Open as a full page” and then at the top right corner of the page click the tree dots and then “Copy link”. If you try to paste this link, it will look something like https://www.notion.so/<your username>/<datatabase id>?v=<view id> - you will need only the database id portion.
Once created, go back to the notebook and click on “Connect”.
General approach
The Notion API works with the json format and when sending a request via the functions of notion-sdk-py, the response will be in json format as well. However, it is not convenient for our purposes to have data in this format, so in the “Initialization” section I added a function (retrieve_db) that converts a retrieved Notion database from json format in a pandas DataFrame. In this way, retrieving all databases is a matter of a single line of code.
Automating recurring tasks
The budget manager, just as a plain Notion page, is already pretty functional as it is… however it would be convenient to save some time and avoid manually doing boring stuff, right?
In the “automation.ipynb” notebook I automated a couple of these tasks, namely renewing subscriptions and managing saving projects.
Renewing subscriptions
The idea here is pretty simple: we’re going to look at the subscriptions database and check whether there are subscriptions that are active and have a next renewal date that is less or equal to the current date. If there are records that match the condition, we are going to renew the subscription.
Practically, I defined a function that works on a single row of a subscriptions data frame that follows 2 main steps:
- Add the subscription renewal as an expense in the expenses database
- Change the last renewal and next renewal dates for the record
Note that both changes do not operate on the pandas data frames we converted but on the Notion databases themselves, we’re therefore going to use a POST operation on the expenses database and an UPDATE operation on the subscriptions database. At the end of the renewal process on all rows, we’re going to retrieve both databases again so we have them in pandas format.
For point 1, we need to define the properties for a new page in json format - remember that properties in this case need to match the properties of the expenses database. I left out the Free expenses relationship property on purpose, since it might be very different for each user and can be easily added either in the code or manually when reviewing expenses.
For point 2 we still have to define properties in json format but this time around it is sufficient to specify the properties that need to be updated - in our case the property “Last renewal” will be set to the value of “Next renewal” and the latter will be set to the current value + the appropriate time delta. To finalise the update we simply call the update function on the corresponding page id giving as arguments the new properties.
IMPORTANT NOTE: if you are a fan of Notion reminders I don’t blame you at all! I use them all the times, especially for reminding me when subscriptions are going to be renewed. However, at the moment, the Notion API doesn’t allow to set reminders, so if you want to set one after renewal you’ll still have to do it manually from Notion - hopefully they will add this feature in the future!
Manage saving projects
One amazing feature that my online bank has, is the ability to create saving projects: in short, these are sections of the bank account in which you can deposit a certain amount of money each month to accomplish some goal, for example, buying a car or another big purchase which you simply can’t afford to pay all at once at the moment. Although the progression of the projects is automated in my bank app, I also like to have an overview on Notion over all my active and completed projects - the only issue is, I have to manually synch the status of the records each time the projects are updated.
Since I’m super lazy, I wrote some code that does it for me 🦥
Again the reasoning is pretty simple: I’ll query the saving projects database (already imported as a pandas data frame) for active projects - if there are active projects I’m further filtering the results for records that have an “Updated on” property that has the month component strictly less than the current month or have the field empty (since new projects might have an empty “Updated on” property).
After this we can define a function that operates on a single row and define the updates for the page in json format:
- The property “Current” will now be
Current + Monthly Amount - The duration of the project in months will be decreased by 1 - if it reaches 0 we’re also going to update the status of the project from “Active” to “Completed”
- Finally we set the field “Updated on” to the current date
After calling the function on all rows, the database has to be retrieved again if you want to work with it.
Generating the dashboard
Finally, the juicy part. How do we generate the home dashboard? We’re going to compute some statistics from the data we have and plot them with plotly. The code for generating the dashboard is in its dedicated notebook, “Home_dashboard.ipynb”.
The summary panel
I wanted to have at the very top of the page, a sort of panel with an overview/recap of the most important things. We can’t do this with plotly, so I leveraged on my basic skills in HTML and CSS to compose the colourful summary panel: the first block simply reports the current month and the date of the last update (useful to check that the code executed correctly), the second one reports some statistics of the current month such as the total income and total expenses, the third one reports statistics on the whole year and finally the last two highlight the top expenses and top income months.
I won’t go too much into detail on how I obtained the statistics since they are simple queries and aggregations from the corresponding databases, instead I’ll focus for a second on how to make the HTML/CSS render properly in the block output.
For the styling part, I defined a simple string, which contains the appropriate CSS styling, that can be formatted with colors taken from the plotly color palettes. I defined the HTML code (in turn, a bunch of divs, nothing fancy) in a separated string formatted with the statistics I previously obtained and then I used the package IPython (more specifically the methods display and HTML) to render the concatenation of the two strings.
A bit of a hack but it worked 😀
The plots
For most of the plots I used plotly express and applied one or more updates on the layout and the traces.
The monthly income partition is the only plot that uses a pie chart representation, simply because I thought it was the most appropriate one to show the partitioning of the monthly income. For the other circular charts I used sunburst instead, as it conveys more information in a single plot - for example in the digital vs. cash yearly overview it is possible to compare not only income and expenses but also which percentage of both was in digital or cash format, additional info can be added in the super useful tooltips that appear when hovering a certain sector.
For the other plots, I preferred to stick with bar charts: for the “Current month free expenses detail” and the “Yearly free expenses overview” I used a single bar representation, using the pattern_shape argument to convey additional info on the mode (digital/cash), moreover I added an horizontal line representing respectively the 100% for the fist chart and the recommended monthly amount for free expenses for the second one. In the second plot I also added text on top of each bar that signals how far (up or down) from the threshold we are for each month.
Finally, the last plot is still a bar chart but this time for each month we have total amounts for different categories as grouped bars and I added a “TOTAL” category at the end to visualize the aggregation on all months.
To include the plots in Notion and arrange them as a dashboard we need to do 3 things:
- Call the method “show” on the plotly figure so it gets rendered in the output of the block
- Select the code block for generating the plot, on the right side of the block click on “Share block”, select “Output” only and copy the link
- Paste the link in the homepage and select “Create embed”. You are free to resize the block and arrange them as you see fit
This feature is really important since it allows to see the plots and interact with them directly in Notion, without opening dedicated web pages or files and the output of the blocks persists even if the notebook is not running and the hardware is offline.
Scheduling the code
The last thing we need to do is scheduling the code so we don’t have to run the notebooks manually every time.
Deepnote has a really convenient way to do this, simply select the notebook, you’ll see at the top of the page a blue button with “Run code”, if you click on the downward arrow next to it you’ll see a “Schedule notebook” option appearing. By clicking on it you can decide a schedule that fits your needs - when you’re done don’t forget to click on “Save schedule” and you’re done! Yay 😊
IMPORTANT NOTE: I discovered this only recently unfortunately, at the moment Deepnote imposes a constraint on scheduling to projects that contain more than one notebook, namely only one notebook can be scheduled per project. So if you want to schedule both but keep them separated you should create a project for each notebook I shared with you. A faster solution, if you want both notebooks to have the same schedule, would be to copy the code of one of them into the other and execute only one.
Conclusion
We’re at the end of our journey in which we set up a functional and semi-automated Notion budget manager! 🥳
I really hope you enjoyed the project and/or you found it useful. Let me know in the comments if you’re using it and if/how you modified it to suit your needs.
See you soon 🦥