Data I Track

Data I track (this is not a complete list, just the things I managed to find/remember so far). I hope to eventually have at least one blog post regarding each one of these. If I get enough blog posts for a single topic, I'll turn it into a category/tag/label and link to it. This post is pinned.

San Jose Parking Garage Data (https://violeteldridge.com/2018/05/26/san-jose-parking-garage-data/) (since 2018-05-22)

  • Health:
    • Basis Peak wristwatch (need to dig around to find out when I got it, but it was totally recalled and service officially shut down on 2016-12-31)
    • Fitbit Charge 2 wristwatch (need to find when I started using it). By creating an Open Humans account and connecting your Fitbit account, you can download/view your data for free online in a Jupyter Notebook. It's not necessary to write any code, because they already wrote the code. All you have to do is run the cells by clicking a cell and typing either shift-return or option-return (or meta-return depending on your operating system). https://www.openhumans.org/activity/fitbit-connection/
    • Symple iPhone health tracker
    • Weight using a Garmin Index Smart Scale (since December 2016)
    • Sleep time using Azumio's iPhone "Sleep Time" app, but unfortunately I don't know of any way to get my raw data out of it - they just let you download the times when you hit the start/stop sleep buttons (since August 2013), which would just be a repeat of what I already have in aTimeLogger
    • Apple Health (just for the functionality of downloading the data from all the applications that integrate with it)
  • Time:
    • aTimeLogger 2 (2013-01 to 2013-03, then consistently from 2013-10 to present (2018-01))
    • RescueTime (since July 2015)
    • Transportation
      • MileLogger (automatic drive capturing) (since December 2017)
      • Waze, but they don't let you download your data and only let you see a very shallow view, limited to the last 30-or-so days, and I don't know when I started because they don't let you download any of your data
  • Money: YNAB (since October 2013); the old iPhone version (which used to be the only version but got renamed to "Classic" when they introduced a new app) that I still have stopped syncing properly a few iOS updates ago. I didn't update because according to reviews, there are still problems with it. Their new app is a subscription model, but I haven't used the budgeting features since 2013 anyways (I only use it for tracking transactions) so I don't see enough value in paying a subscription for this when I could get functionality equivalent to what I use from the application by using an Excel spreadsheet. So now I only use the desktop version.
  • Weather:
  • Computer (since January 2016):
    • every second, current application and title of current window (since December 2017) (see blog post Personal RescueTime (Computer activity time tracker))
    • every hour
      • `uptime`
      • firefox processes: `ps aux | grep -i [f]irefox`
      • memory usage of windowserver (windowing system on mac computers)
    • every 2.5 hours: network and location: public IP, gps coordinates, output of `/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport -I` and `ifconfig`
    • every 8 hours: disk usage: `df -h .`
    • every day: disk size of iPhone backup: `echo $(du -s ~/Library/Application\ Support/MobileSync/) $(du -hs ~/Library/Application\ Support/MobileSync/)
    • battery status `pmset -g batt`, every hour 2017-07-26--2018-03-04, every 10 minutes 2018-03-05-present
  • Other:
    • Nomie 2 https://nomie.io/ (makes it easy to manually track many little things) (since January 2018)
    • Family AT&T Data Usage (since January 2017)
    • Things I track (i.e. this blog post, since 2018-01-15)

I'm interested in figuring out whether it's possible for someone with a Garmin fitness tracker that tracks location data to be able to download the raw location data. I would expect it to be a list of latitude/longitude with their respective timestamps, every few seconds or minute depending on the situation.
Another possibility is also to use a USB data tracker like https://www.amazon.com/Elitech-RC-5-Temperature-Recorder-Accuracy/dp/B00MQSCZF2/.
And a weather station like https://www.amazon.com/Ambient-Weather-WS-2902-Professional-Monitoring/dp/B01N5TEHLI; it says it's $30.01 off, down from $180.00 to $149.99 on 2018-01-21. (see price history at CamelCamelCamel.com)

Keep time while reading a book

I wrote this little script to help me keep on track with my reading. I needed to read 2 chapters of each of 3 different books today, and I wanted to assign one hour to each chapter, but in order to keep on track, I wanted to see what page I should be on at each moment, so that's why I wrote this script.

It's in python. Just tell it the amount of time or the time you want to finish (e.g. '1hour' or '2pm', anything that is supported by the gnu date `-d` argument), the page you're starting on, and the page you're finishing on, and it will continuously tell you where you should be in your reading.

https://github.com/mica5/reading-progress

Why I sleep how/when I do

People are often surprised by my sleep schedule, especially since it's considered to be the "opposite" of people in a similar situation as my own (Software Engineer, University student).

I need about 8 hours of sleep each night to feel my best each day. I don't set an alarm unless I go to bed late and need to get up early and haven't been getting enough sleep for some days in a row.

I'm more likely to go to sleep at regular time on weekend nights (e.g. Friday and Saturday) if I'm doing something I don't particularly enjoy (e.g. reading or writing for a class I don't want to take but have to take), rather than doing something I enjoy such as reading or doing work for a class I do enjoy.

By going to bed 9 hours before I need to be up, even if I'm in bed 20 minutes late, that still gives me an extra 40 minutes for sleep. Allowing myself an extra 20-60 minutes for sleep also allows my body to get extra sleep if it needs it, which I sometimes do.

Since I'm usually out of bed before 4:40am, I can do my morning routine and be out of the house by 5:40am. In my experience, the number of cars on the road sharply increases starting 5 minutes before 6am, so leaving at 5:40am usually means there's still empty lanes on the freeway.

By getting to my destination approximately 3 hours before anyone else does, that's nearly 3 hours of focused, uninterrupted work time.

Maybe someday I'll support all of this with data and visualizations. I have a lot of data for this, I just need some time to analyze it.

Miscellaneous

2018-01-01 Here are some ideas that don't have enough content to be individual blog posts, so I'll write here until they become developed enough to warrant their own posts.

<h1>
Python</h1>
2018-01-01 "One of the curses of Python is that it is such an easy and expressive language that developers often find it easier to re-invent and re-implement functionality that exists in other packages than work to integrate code from other packages." <a href="http://www.aosabook.org/en/matplotlib.html">http://www.aosabook.org/en/matplotlib.html</a> (my attention was brought to this post as a result of course 2/5 "<a href="https://www.coursera.org/learn/python-plotting">Python Plotting</a>" in the Coursera specialization "<a href="https://www.coursera.org/specializations/data-science-python">Applied Data Science with Python</a>")

2018-01-01 Compared to typed languages, Python will let you do anything, which allows inexperienced programmers (and experienced ones) to cause run-time errors in user-facing systems, but also provides an expressive language in which a masterpiece can be written.

<h1>
Internet and search engines</h1>
2018-01-01 One of the great things about the internet is that many people can say the same thing in many different ways. Consider the concept of pointers from the C language - you can pass around references. Just remember where the original is and make pointers back. On the internet, there exist many ideas, but many people use different language to search for or ask about the same ideas. Many forum or stackoverflow answers show "this is a duplicate, see [this other url here with the answer]". But the benefit of that is that there are many pointers laying around the internet that allow many users who search using different language and terminology to find the same answer by many of the pointers that are laying around. I think that could make it much easier for AI-/NLP- based search engines to connect content together with many variations of the same language.

Personal RescueTime (Computer activity time tracker)

I have been a paid user of RescueTime every month since June 2015, but since I love collecting and analyzing data, I decided to replicate the functionality and keep track of the data in my own local database. Using applescript and postgresql, I've been recording computer usage (application name and window title) since "2017-12-08 20:06:00.670021". I was just completing a survey on Coursera, which is asking "How many hours per week do you plan to spend working on this course?", and because I started recording data on December 8th and started taking the class on the 6th, I knew I had data for the majority of my coursera time. This doesn't account for course-related material that wasn't on coursera's website or didn't have "Coursera" in the title, and now I'm also realizing I should try to grab a url of the current application I'm in, if I can (unfortunately, some quick google searches don't return anything obvious for firefox, although chrome does have it built in..).


with base as (
    select
            *
            , datetime-lag(datetime) over (partition by true order by datetime) as time_between
            , extract(year from datetime)::text||'-'||extract(week from datetime)::text as week
        from user_activities
        where (window_name ilike '%coursera%')
            or (application ilike '%firefox%' and (
                window_name ilike '%week+%'
                or window_name ilike '%assignment%'))
        order by datetime
    ) , filtered as (
        select *
        from base
        where time_between
week   | time_spent_on_coursera
---------+------------------------
 2017-49 | 00:48:47.559574
 2017-50 | 09:45:30.929945
 2017-51 | 04:02:11.548172
(3 rows)

Note that for week 2017-50, that's 9 hours and 45 minutes. If anyone is interested, I could come back and explain this a bit. It's not completely accurate, because Coursera refers to some materials that don't have "coursera" in the title, and I could have been busy doing something non-coursera-related for less than 15 minutes, which would make it less accurate.

I recommend RescueTime, as it is an awesome tool for finding out a quantitative truth about where you spend your time on the computer: https://www.rescuetime.com

Useful bash commands

Remove the last 1 line of output:

printf "1\n2\n3\n4\n5\n" | head -n-1
1
2
3
4

without " | head -n-1", the printf command yeilds:

printf "1\n2\n3\n4\n5\n"
1
2
3
4
5

Remove the first 1 line of output:

printf "1\n2\n3\n4\n5\n" | tail -n+2
2
3
4
5

You can change the number in any command you like. tail -n+1 doesn't seem to remove any lines, so add 1 to whatever number of lines you want to remove.

Coffee Notes

Forager (in San Jose) opened late February 2017.

Devout Coffee brand, San Sebastian beans - from Guatemala, natural process.


Coffee Classes at Chocolate Fish Coffee Roasters in Sacramento at 4749 Folsom Boulevard, East Sacramento.

Coffee books/authors:

2017-12-18
In coffees I look for fruity flavors. Usually when coffee people say they taste specific fruit flavors like nectarine, strawberries, etc, I don't usually taste it. The only kind-of fruit flavor I have ever really been able to taste in a coffee was blueberries in a coffee from Ethiopia (unfortunately, I don't remember which one specifically).

The best coffee I've ever had was Guji Highland by Devout, but unfortunately, its season is over. I don't know whether it will have another season in the future. Forager Tasting Room and Eatery in Downtown San Jose uses their beans, but I'm not sure if they sell whole beans. Another really good coffee I'm having lately is San Sebastian, also by Devout. Other good coffee bean producers are Verve (Santa Cruz) and Temple (Sacramento). I haven't enjoyed any coffees from Allegro, which is too bad, because Whole Foods carries a lot of their beans. Starbucks coffees mostly taste like burnt coal, and Philz Coffee "Lighter Blend" "Sooo Good" also tastes burnt.

Raspberry Pi plot of temperature/humidity/pressure data

Improvements to be made:

  • convert x axis to utc time and make ticks at every hour or every few hours
  • add: light intensity sensor (for sun), rain gauge, wind direction/speed
  • move the temperature sensor out of sunlight to reduce effects from radiant heat, and move it farther away from the window to reduce influence from indoor temperature
  • put pressure on its own axis so it doesn't need to be divided
  • add units to everything (pressure is in hPa)
  • list everything it took to build my setup

Parts

Raspberry Pi 3

MCM Electronics Raspberry Pi 3 Project Kit with GPIO Breakout, Breadboard and Components (On Amazon, which reports it for about $87.99, but it was at Fry's for $82.99). This is a great starter kit, but is unsupported. The company that made it, MCM Electronics, was apparently bought or merged, and whoever is now in control chose to no longer support the users of this kit, because the starter guide that a piece of paper in the kit refers to doesn't exist anywhere on MCM's website (or anywhere on the internet; I searched for at least 20 minutes to no avail).

Humidity/Temperature Sensor

On Adafruit.com

Pressure sensor

On ControlEverything.com

Waterproof temperature sensors

Vktech 5pcs 2M Waterproof Digital Temperature Temp Sensor Probe DS18b20, on Amazon.


A great tutorial of how to use the temperature sensors: https://www.jeremymorgan.com/tutorials/raspberry-pi/monitor-room-temperature-raspberry-pi/

Where to find more Raspberry-Pi-like computers: https://www.digikey.com/products/en/development-boards-kits-programmers/evaluation-boards-embedded-mcu-dsp/786. I haven't looked through it thoroughly, but my brother told me about it. Raspberry Pi seems to be the one that gets the press and is made as a starter kit for beginners, and possibly even started the idea (don't quote me on that; this is just speculation).

My LinkedIn profile links to this page: https://www.linkedin.com/in/mica-eldridge-510339b9

iPhone: Silence an incoming call without letting the caller know

I get a lot of spam calls, but want to silence the ringer without tipping the caller off that I did it. If you want to make an incoming call stop ringing without answering or letting the caller know you hung up, then press one of the volume buttons. If you press the menu button, power button, or on-screen "Decline" button, then the caller will immediately be sent to voicemail (or whatever is in its place), and if it happens soon enough after they dialed you, they will know that you intentionally declined their call. If you use one of the sound buttons, then the caller will continue ringing, but your phone will stop making the ringing sound.