OT? want to save web page image automatically

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

OT? want to save web page image automatically

geek
I have a web page that displays a chart of values that change once per
hour. It it uses chart.js under node.js on mint.

See it here: http://209.145.101.163/24chart.html

If I right click on the image I have an option (in Firefox) to save it
as a png. That works fine. What I'd like to do is save the image once
per hour without intervention. Does anyone know a scriptable method?

TIA

Dave

--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
Reply | Threaded
Open this post in threaded view
|

Re: OT? want to save web page image automatically

David Fletcher-5
On Mon, 2020-11-09 at 16:40 -0800, Dave Stevens wrote:

> I have a web page that displays a chart of values that change once
> per
> hour. It it uses chart.js under node.js on mint.
>
> See it here: http://209.145.101.163/24chart.html
>
> If I right click on the image I have an option (in Firefox) to save
> it
> as a png. That works fine. What I'd like to do is save the image once
> per hour without intervention. Does anyone know a scriptable method?
>
> TIA
>
> Dave

Hi Dave,

That looks good. Is it particulate matter you are charting?

One of my "lock-down projects" was to get my outside air temperature
sensor working to my satisfaction, see

https://flightmaker.hopto.org/airtemperature.shtml

I did a little searching and found a nice single supply rail
instrumentation amp that did the trick for me.

I'm simply keeping a log file of my readings in CSV format with UTC
time like this
2020,11,10,07,15,11.0
2020,11,10,07,30,11.2
2020,11,10,07,45,11.1
2020,11,10,08,00,11.2
2020,11,10,08,15,11.4
2020,11,10,08,30,11.5

Would it not be better to just log your data like this so you can run
whatever analysis you like with it any time in the future? Mine is just
a hobby but I might be looking into doing some charting in the future.
I know storage is cheap these days but CSV takes a lot less and is
surely more flexible.

Dave


--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
Reply | Threaded
Open this post in threaded view
|

Re: OT? want to save web page image automatically

Colin Law
In reply to this post by geek
On Tue, 10 Nov 2020 at 00:42, Dave Stevens <[hidden email]> wrote:
>
> I have a web page that displays a chart of values that change once per
> hour. It it uses chart.js under node.js on mint.
>
> See it here: http://209.145.101.163/24chart.html
>
> If I right click on the image I have an option (in Firefox) to save it
> as a png. That works fine. What I'd like to do is save the image once
> per hour without intervention. Does anyone know a scriptable method?

Rather than do it like that you would get a much more flexible system
using node-red in conjunction with influxdb to store the data and
grafana to show it.  Then you would be able to easily review
historical data, zoom in, etc.

Health Warning:  node-red is highly addictive and may well have a
detrimental effect on your personal relationships.  There is an
excellent help group for those suffering from this at
https://discourse.nodered.org, though the chances of long term
recovery are slim.

Colin

--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
Reply | Threaded
Open this post in threaded view
|

Re: OT? want to save web page image automatically

geek
In reply to this post by David Fletcher-5
On Tue, 10 Nov 2020 08:49:56 +0000
David Fletcher <[hidden email]> wrote:

> On Mon, 2020-11-09 at 16:40 -0800, Dave Stevens wrote:
> > I have a web page that displays a chart of values that change once
> > per
> > hour. It it uses chart.js under node.js on mint.
> >
> > See it here: http://209.145.101.163/24chart.html
> >
> > If I right click on the image I have an option (in Firefox) to save
> > it
> > as a png. That works fine. What I'd like to do is save the image
> > once per hour without intervention. Does anyone know a scriptable
> > method?
> >
> > TIA
> >
> > Dave  
>
> Hi Dave,
>
> That looks good. Is it particulate matter you are charting?
>
> One of my "lock-down projects" was to get my outside air temperature
> sensor working to my satisfaction, see
>
> https://flightmaker.hopto.org/airtemperature.shtml
>
> I did a little searching and found a nice single supply rail
> instrumentation amp that did the trick for me.
>
> I'm simply keeping a log file of my readings in CSV format with UTC
> time like this
> 2020,11,10,07,15,11.0
> 2020,11,10,07,30,11.2
> 2020,11,10,07,45,11.1
> 2020,11,10,08,00,11.2
> 2020,11,10,08,15,11.4
> 2020,11,10,08,30,11.5
>
> Would it not be better to just log your data like this so you can run
> whatever analysis you like with it any time in the future? Mine is
> just a hobby but I might be looking into doing some charting in the
> future. I know storage is cheap these days but CSV takes a lot less
> and is surely more flexible.
>
> Dave
>
>

I do in fact keep readings taken about once per second then datamash
them to get an hourly. All raw readings are kept as csvs like:

2020-11-1011:00:01 concentration_pm2.5_atmos: 2 µg/m^3
2020-11-1011:00:02 concentration_pm2.5_atmos: 2 µg/m^3
2020-11-1011:00:04 concentration_pm2.5_atmos: 2 µg/m^3
2020-11-1011:00:06 concentration_pm2.5_atmos: 2 µg/m^3
2020-11-1011:00:07 concentration_pm2.5_atmos: 2 µg/m^3
2020-11-1011:00:09 concentration_pm2.5_atmos: 2 µg/m^3
2020-11-1011:00:11 concentration_pm2.5_atmos: 2 µg/m^3
2020-11-1011:00:13 concentration_pm2.5_atmos: 2 µg/m^3
2020-11-1011:00:14 concentration_pm2.5_atmos:   2 µg/m^3
2020-11-1011:00:16 concentration_pm2.5_atmos: 2 µg/m^3

I'd be interested in discussing the design more but perhaps off-list.

D


--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
Reply | Threaded
Open this post in threaded view
|

Re: OT? want to save web page image automatically

Peter Flynn
On 10/11/2020 20:58, Dave Stevens wrote:

> On Tue, 10 Nov 2020 08:49:56 +0000
> David Fletcher <[hidden email]> wrote:
>
>> On Mon, 2020-11-09 at 16:40 -0800, Dave Stevens wrote:
>>> I have a web page that displays a chart of values that change once
>>> per
>>> hour. It it uses chart.js under node.js on mint.
>>>
>>> See it here: http://209.145.101.163/24chart.html
>>>
>>> If I right click on the image I have an option (in Firefox) to save
>>> it
>>> as a png. That works fine. What I'd like to do is save the image
>>> once per hour without intervention. Does anyone know a scriptable
>>> method?

AFAIK you can run FF from the command line with an option to open a page
and screenshot it.

https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Headless_mode

Basically, create a special profile first (in about:profiles), and then

$ firefox -P <profilename> --screenshot webpage.jpg https://whatever/ --exit

This creates a (potentially VERY tall, narrow) screenshot of the whole
page, exactly like Vivaldi's pageshot button does. Excellent for making
screenshot for documentation.

Peter

--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
Reply | Threaded
Open this post in threaded view
|

Re: OT? want to save web page image automatically

Karl Auer
On Tue, 2020-11-10 at 21:52 +0000, Peter Flynn wrote:

> On 10/11/2020 20:58, Dave Stevens wrote:
> > On Tue, 10 Nov 2020 08:49:56 +0000
> > David Fletcher <[hidden email]> wrote:
> >
> > > On Mon, 2020-11-09 at 16:40 -0800, Dave Stevens wrote:
> > > > I have a web page that displays a chart of values that change
> > > > once
> > > > per
> > > > hour. It it uses chart.js under node.js on mint.
> > > >
> > > > See it here: http://209.145.101.163/24chart.html
> > > >
> > > > If I right click on the image I have an option (in Firefox) to
> > > > save
> > > > it
> > > > as a png. That works fine. What I'd like to do is save the
> > > > image
> > > > once per hour without intervention. Does anyone know a
> > > > scriptable
> > > > method?
>
> AFAIK you can run FF from the command line with an option to open a
> page
> and screenshot it.
>
> https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Headless_mode
>
> Basically, create a special profile first (in about:profiles), and
> then
>
> $ firefox -P <profilename> --screenshot webpage.jpg https://whatever/
>  --exit
>
> This creates a (potentially VERY tall, narrow) screenshot of the
> whole
> page, exactly like Vivaldi's pageshot button does. Excellent for
> making
> screenshot for documentation.
>
> Peter
>

If you want an actual inage or other discrete element off a page, you
can use crl or wget to retrieve it very simply.

In this particular case it's JavaScript output, so not quite so simple,
though you CAN download the JS easily enough, and it onl;ty uses one
external script (Chart) which is also easy to grab.

There would be a little JS programming effort to wrap it up into
something locally executable, and a little more to append one chart to
the previous chart or do some kind of history. Depends whether you are
ever interested in old charts, or only ever want the current one.
Haven't ever used node-red, maybe it could do this - of course this
would mean sliding into addiction followed by inevitable death.

If you really do only want the image (not the liveness of the chart),
then Peter's idea of grabbing a screenshot would be the starting point,
followed by using imagemagick or similar to chop out just the image
(which is at the same position on the page every time).

Or you could write to the author and ask for a static chart below his
live chart, so you can just grab the static image with curl or wget.

Regards, K.

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Karl Auer ([hidden email])
http://www.biplane.com.au/kauer

GPG fingerprint: 2561 E9EC D868 E73C 8AF1 49CF EE50 4B1D CCA1 5170
Old fingerprint: 8D08 9CAA 649A AFEF E862 062A 2E97 42D4 A2A0 616D




--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users