I wanted to quickly write on this before it slipped my mind and because the answer was not so easy to find in the net.
For whatever reason, when sending data to Graphite, there has been an error. So one or more data points are wrong. For the sake of this post, it doesn't matter how and from where we are sending the data (sensu, directly to Graphite ...), what matters is that we have sent wrong data and we want to get it right. In the example below, we didn't send the total number of rentals in Italy for the Aug 8th, so the graph shows no data for that day:
We login to our Graphite server and we go to the location of the whisper database that we want to update. Typically it will be something like:
/opt/graphite/storage/whisper/[path to your metric]/[metric name].wsp
In our example above the whisper database that we want to update is at:
We will use two whisper commands to achieve our objective. whisper-fetch.py to check the data in the database and whisper-update.py to actually correct the data.
First of all we need to pinpoint which is the data-point that we need to update. Data-points are stored in whisper as a pair composed of Epoch time-stamp and value.
In the example above, we can see that the wrong data point was sent to Graphite on Aug 7th at 02:00. So let's try to identify this:
root@graphite:/opt/graphite/storage/whisper/rentals# whisper-fetch.py --from=1407196800 --until=1407542400 rentals_per_country.wsp 1407283200 137.000000 1407369600 0.000000 1407456000 146.000000 1407542400 145.000000
Here we have used whisper-fetch to get the data points from Aug 6th to Aug 9th (so including Aug 7th that is our incorrect data-point). The from and until parameters are Aug 6th and Aug 9th expressed in Epoch time. You can easily convert dates from human time to Epoch time here.
We can clearly see the wrong data-point in the list above and it is uniquely identified by 1407369600 Epoch time-stamp.
So now we just need to update this specific data-point with what we deem to be the correct value. We use whisper-update for this:
root@graphite:/opt/graphite/storage/whisper/rentals# whisper-update.py rentals_per_country.wsp 1407369600:140.000000
And we can recheck our graph and problem gone!