How to script at job creation?

[Bo Berglund]

I am running an Ubuntu 18.04 LTS server where I want to have some
commands run at specific times. So I have used at to create them
interactively inside a terminal window. The number of jobs each day is
12 or more and they are all very similar but not identical.

So now I am repeating the same chore every day over and over and it
feels like hacking away at the keyboard is not productive, so I wonder
if there is some way to *script* the creation of at jobs?

I need at rather than cron because the at job creation saves the user
and the current dir and it seems like also the user environment at the
time of creation.
So the job will run just like I was inside the target dir and executed
the job on the terminal command line.

With cron it gets much more complex...

The jobs I want to run call my scripts (located in $USER/bin) so this
must work, and it does when I create the jobs manually.

So I would like to create a script I can run in the target dir and it
will create the 12 different at jobs in one command execution.

Can this be done at all?


--
Bo Berglund
Developer in Sweden


--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

[Karl Auer]

On Thu, 2020-12-10 at 11:18 +0100, Bo Berglund wrote:
> I need at rather than cron because the at job creation saves the user
> and the current dir and it seems like also the user environment at
> the time of creation.

Each user has a crontab, and in the main crontab you can specify a
user, so that's half the "problem" solved. You can change to any
desired directory as part of running any cronjob, so that part of the
problem is also solved. What specifically do you need from the user
environment?

> With cron it gets much more complex...

Generally not, unless the job needs interactive input, which would be a
problem with at too, so...

> So I would like to create a script I can run in the target dir and it
> will create the 12 different at jobs in one command execution.
>
> Can this be done at all?

Absolutely. If all the scripts are run as the same user, you only need
one master script, and it simply runs all the other scripts. Run the
master script from cron. If any of the scripts have output, capture it
with redirection to a file. If the scripts take a long time to run, you
might like to run then simultaneously by adding "&" at the end of each
individual script invocation. Or capture all the output from all the
scripts by redirecting the master script's output.

You might have to twiddle it because it is off the top of my head, but
a line something like this in crontab will run master.sh:

- every day at 5:05AM
- as fred
- from fred's home directory
- after changing to /opt
- with MYVAR set to the value "thingy":

5 5 * * * fred ( cd /opt ; export MYVAR=thingy ; ~/master.sh ) > 2>
/tmp/log.txt 2>&1

Or put the same line, minus the "fred", into fred's crontab.

And of course you can also pass parameters to the script, or load
information from files or do whatever else a script, running as fred,
could normally do.

If the scripts are not supposed to all run at the same time, you can
either put a suitable "sleep" command between the lines in the master
script or (much better) have one cron job file for each script.

Regards, K.

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Karl Auer ([hidden email])
http://www.biplane.com.au/kauer

GPG fingerprint: 2561 E9EC D868 E73C 8AF1 49CF EE50 4B1D CCA1 5170
Old fingerprint: 8D08 9CAA 649A AFEF E862 062A 2E97 42D4 A2A0 616D




--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

[Karl Auer]

On Thu, 2020-12-10 at 22:45 +1100, Karl Auer wrote:
> If the scripts are not supposed to all run at the same time, you can
> either put a suitable "sleep" command between the lines in the master
> script or (much better) have one cron job file for each script.

That should read "one cron job for each script". I.e. one line in
crontab for each script.

Regards, K.

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Karl Auer ([hidden email])
http://www.biplane.com.au/kauer

GPG fingerprint: 2561 E9EC D868 E73C 8AF1 49CF EE50 4B1D CCA1 5170
Old fingerprint: 8D08 9CAA 649A AFEF E862 062A 2E97 42D4 A2A0 616D




--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

[Bo Berglund]

On Thu, 10 Dec 2020 11:18:52 +0100, Bo Berglund
<[hidden email]> wrote:

>So I would like to create a script I can run in the target dir and it
>will create the 12 different at jobs in one command execution.
>
>Can this be done at all?

I found a way to get rid of the user interaction when creating an at
job as follows (all on one line):

echo "timeout --signal=2 65m getstream input18.mp4" | at 23:58
tomorrow

When executed on the command line it creates the at job without any
interaction at all, which was my first hurdle.

I will test putting a few of these commands into a script, which I can
then run daily to create the at jobs.
I probably should put a unique datestamp on the output mp4 file of
course, but that can be done in the script itself.

The getstream script packages a call to youtube-dl to download from a
live stream and the timeout of 65 minutes assures that a full hour is
available in the resulting file.
Without the timeout call youtube-dl continues downloading the stream
for 6h00m, which is not what I want.


--
Bo Berglund
Developer in Sweden


--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

[Bo Berglund]

On Thu, 10 Dec 2020 14:05:01 +0100, Bo Berglund
<[hidden email]> wrote:

>I found a way to get rid of the user interaction when creating an at
>job as follows (all on one line):
>
>echo "timeout --signal=2 65m getstream input18.mp4" | at 23:58
>tomorrow
>
>When executed on the command line it creates the at job without any
>interaction at all, which was my first hurdle.
>
>I will test putting a few of these commands into a script, which I can
>then run daily to create the at jobs.

I must be missing something important because I cannot get this to
work from within a script...
This is what I have now and it just refuses to *run* the echo command
inside the terminal.

#!/bin/bash
NEXTDAY=`date --date="tomorrow" +%Y-%m-%d`
CMDTIM1="echo \"timeout --signal=2 65m getmsnbcstream ${NEXTDAY}_"
#Now create the at jobs
CMDAT="${CMDTIM1}inp19.mp4\" | at 00:58 tomorrow"
$CMDAT
exit

What happens when I run this from the command line is:

$ ./makeatjobs
"timeout --signal=2 65m getmsnbcstream 2020-12-12_inp19.mp4" | at
00:58 tomorrow

I.e. it just *displays* the command I want to execute with echo and
pipe into at after removing "echo"...
So why is echo removed from the command (seems like the script
*executes* echo internally rather than sending it to the command
line)?

In other scripts I have written this does not happen, there the
command string gets executed as expected with the same basic script
structure. But then I don't use echo...

So is there a special handling involved when one wants to use echo to
pipe a string over into another command?

Notice that the echo command itself disappears and the only thing
happening is that what follows echo is displayed on the terminal.

How does one execute the echo command itself including the pipe to the
second process?
I need the script to do the following, which works if I type it into
the command window manually:

echo "timeout --signal=2 65m getmsnbcstream 2020-12-12_inp19.mp4" | at
00:58 tomorrow


--
Bo Berglund
Developer in Sweden


--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

[Volker Wysk]

Am Freitag, den 11.12.2020, 08:12 +0100 schrieb Bo Berglund:
> #!/bin/bash
> NEXTDAY=`date --date="tomorrow" +%Y-%m-%d`
> CMDTIM1="echo \"timeout --signal=2 65m getmsnbcstream ${NEXTDAY}_"
> #Now create the at jobs
> CMDAT="${CMDTIM1}inp19.mp4\" | at 00:58 tomorrow"
> $CMDAT
> exit

Aren't there two closing quotes missing, in lines 3 and 5 ...?

Bye
Volker

--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

[Karl Auer]

On Fri, 2020-12-11 at 08:12 +0100, Bo Berglund wrote:
> #!/bin/bash
> NEXTDAY=`date --date="tomorrow" +%Y-%m-%d`
> CMDTIM1="echo \"timeout --signal=2 65m getmsnbcstream ${NEXTDAY}_"
> #Now create the at jobs
> CMDAT="${CMDTIM1}inp19.mp4\" | at 00:58 tomorrow"
> $CMDAT
> exit

You are hiding shell control from the current shell. To fix, just put
"eval" at the start of the second-last line:

eval $CMDAT

Read "man bash" for a description of what eval does - basically the
arguments to it become a shell command again (instead of a string
argument to echo, which is what you've ended up with).

There is no need to build commands like that unless you do actually
want to echo them. Here is an alternative (untested!):

#!/bin/bash
PREFIX=`date --date="tomorrow" +%Y-%m-%d`
SUFFIX="inp19.mp4"
FILE="$PREFIX$SUFFIX"

ATINPUT="timeout --signal=2 65m getmsnbcstream $FILE"
AT="at 00:58 tomorrow"
echo $ATINPUT | $AT

BTW if you want to see what commands are executing put "-x" at the end
of the shebang line ("#!/bin/bash -x"). To see just some lines, put
"set -x" in front of and "set +x" after the section you want to see.

Also, there is no need to explicitly exit a shell script unless you
want to set a specific return value. The script will exit with the
return value of the most recently executed command.

Regards, K.

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Karl Auer ([hidden email])
http://www.biplane.com.au/kauer

GPG fingerprint: 2561 E9EC D868 E73C 8AF1 49CF EE50 4B1D CCA1 5170
Old fingerprint: 8D08 9CAA 649A AFEF E862 062A 2E97 42D4 A2A0 616D




--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

[Bo Berglund]

On Fri, 11 Dec 2020 19:19:34 +1100, Karl Auer <[hidden email]>
wrote:

>On Fri, 2020-12-11 at 08:12 +0100, Bo Berglund wrote:
>> #!/bin/bash
>> NEXTDAY=`date --date="tomorrow" +%Y-%m-%d`
>> CMDTIM1="echo \"timeout --signal=2 65m getmsnbcstream ${NEXTDAY}_"
>> #Now create the at jobs
>> CMDAT="${CMDTIM1}inp19.mp4\" | at 00:58 tomorrow"
>> $CMDAT
>> exit
>
>You are hiding shell control from the current shell. To fix, just put
>"eval" at the start of the second-last line:
>
>eval $CMDAT

Thanks!
This is what solved the problem.
Seems similar to the Windows exec command.

>
>There is no need to build commands like that unless you do actually
>want to echo them. Here is an alternative (untested!):

The only way I found by searching the net in order to have a 1-line
command to set an at task was to use the echo trick and pipe the
output into at as shown.

Since I have 16 such at tasks to create I wanted to write a script for
it.
And then I got into the echo problem...
What I showed here was of course only the first call...


>echo $ATINPUT | $AT

I guess you forgot the eval keyword here... :)

>Also, there is no need to explicitly exit a shell script unless you
>want to set a specific return value. The script will exit with the
>return value of the most recently executed command.

My exit was here in order to stop the script after the first command
when testing.
I do not have an exit otherwise.

Thanks again!

--
Bo Berglund
Developer in Sweden


--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

[Bo Berglund]

On Fri, 11 Dec 2020 08:29:33 +0100, Volker Wysk <[hidden email]>
wrote:

>Am Freitag, den 11.12.2020, 08:12 +0100 schrieb Bo Berglund:
>> #!/bin/bash
>> NEXTDAY=`date --date="tomorrow" +%Y-%m-%d`
>> CMDTIM1="echo \"timeout --signal=2 65m getmsnbcstream ${NEXTDAY}_"
>> #Now create the at jobs
>> CMDAT="${CMDTIM1}inp19.mp4\" | at 00:58 tomorrow"
>> $CMDAT
>> exit
>
>Aren't there two closing quotes missing, in lines 3 and 5 ...?
>

No, these are 2 quotes that are explicitly escaped using \" because
they need to get to the output string.


--
Bo Berglund
Developer in Sweden


--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

[Karl Auer]

On Fri, 2020-12-11 at 11:10 +0100, Bo Berglund wrote:
> > eval $CMDAT
>
> Thanks!
> This is what solved the problem.
> Seems similar to the Windows exec command.

There is a bash exec command too - eval runs the command in a subshell
and returns, exec replaces the running process with the new one.

> The only way I found by searching the net in order to have a 1-line
> command to set an at task was to use the echo trick and pipe the
> output into at as shown.

You could make a function that takes the changing components and
executes at... then each "script" is a one-liner calling the function.

> > echo $ATINPUT | $AT
>
> I guess you forgot the eval keyword here... :)

Doesn't need it.

Regards, K.

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Karl Auer ([hidden email])
http://www.biplane.com.au/kauer

GPG fingerprint: 2561 E9EC D868 E73C 8AF1 49CF EE50 4B1D CCA1 5170
Old fingerprint: 8D08 9CAA 649A AFEF E862 062A 2E97 42D4 A2A0 616D




--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

[Colin Watson]

On Fri, Dec 11, 2020 at 08:12:54AM +0100, Bo Berglund wrote:

> I must be missing something important because I cannot get this to
> work from within a script...
> This is what I have now and it just refuses to *run* the echo command
> inside the terminal.
>
> #!/bin/bash
> NEXTDAY=`date --date="tomorrow" +%Y-%m-%d`
> CMDTIM1="echo \"timeout --signal=2 65m getmsnbcstream ${NEXTDAY}_"
> #Now create the at jobs
> CMDAT="${CMDTIM1}inp19.mp4\" | at 00:58 tomorrow"
> $CMDAT
> exit

Something like the following approach makes the quoting much easier to
follow.  I recommend it instead of using "eval" or similar approaches.

#! /bin/bash
NEXTDAY="$(date --date=tomorrow +%Y-%m-%d)"
at 00:58 tomorrow <<EOF
timeout --signal=2 65m getmsnbcstream ${NEXTDAY}_inp19.mp4
EOF

--
Colin Watson (he/him)                              [[hidden email]]

--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

[Gary Aitken]

On 12/11/20 12:29 AM, Volker Wysk wrote:> Am Freitag, den 11.12.2020, 08:12 +0100 schrieb Bo Berglund:
>> #!/bin/bash
>> NEXTDAY=`date --date="tomorrow" +%Y-%m-%d`
>> CMDTIM1="echo \"timeout --signal=2 65m getmsnbcstream ${NEXTDAY}_"
>> #Now create the at jobs
>> CMDAT="${CMDTIM1}inp19.mp4\" | at 00:58 tomorrow"
>> $CMDAT
>> exit
>
> Aren't there two closing quotes missing, in lines 3 and 5 ...?

No, the middle one in each case is escaped.

Gary


--
ubuntu-users mailing list
[hidden email]
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users