Warning, /education/kstars/doc/ekos-scheduler.docbook is written in an unsupported language. File is not indexed.

0001 <sect2 id="ekos-scheduler">
0002     <title>Scheduler</title>
0003     <indexterm>
0004         <primary>Tools</primary>
0005         <secondary>Ekos</secondary>
0006         <tertiary>Scheduler</tertiary>
0007     </indexterm>
0008     <screenshot>
0009         <screeninfo>
0010             Ekos Scheduler Module
0011         </screeninfo>
0012         <mediaobject>
0013             <imageobject>
0014                 <imagedata fileref="ekos_scheduler.png" format="PNG"/>
0015             </imageobject>
0016             <textobject>
0017                 <phrase>Ekos Scheduler Module</phrase>
0018             </textobject>
0019         </mediaobject>
0020     </screenshot>
0021     <sect3 id="ekos-scheduler-introduction">
0022         <title>Introduction</title>
0023         <para>
0024             Ekos Scheduler is an indispensable arsenal in building your robotic observatory. A Robotic observatory is an observatory composed of several subsystems that are orchestrated together to achieve a set of scientific objectives without human intervention. It is the only Ekos module that does not require Ekos to be started as it is utilized to <emphasis>start</emphasis> and <emphasis>stop</emphasis> Ekos. It is designed to be straightforward and intuitive. However, the scheduler should only be used after you mastered Ekos and knows all the quirks of your equipment. Since the complete process is automated, including focus, guiding, and meridian flip, all equipment should be thoroughly used with Ekos and all their parameters and settings adjusted to achieve the best result.
0025         </para>
0026         <para>
0027             With Ekos, the user can utilize the powerful sequence queue to image batches of images for a particular target. In simple setups, the user is expected to focus the CCD, align the mount, frame the target, and start guiding before initiating the capture process. For more complex observatory environments, there are usually predefined custom procedures to be executed to prepare the observatory for imaging, and another set of procedures on shutdown. The user may plan to image one or more targets during the night and expects data to be ready by morning. In &kstars;, tools such as the <link linkend="tool-obsplanner">Observation Planner</link> and <link linkend="tool-whatsup">What's up Tonight</link> help the user in selecting candidates for imaging. After selecting the desired candidates, the user can add them to the Ekos Scheduler list for evaluation. The user may also add the targets directly in Ekos scheduler or select a FITS file of a previous image.
0028         </para>
0029     </sect3>
0030 
0031     <sect3 id="ekos-scheduler-settings">
0032         <title>Settings</title>
0033         <para>
0034             Ekos Scheduler provides a simple interface to aid the user in setting the conditions and constraints required for an observation job. Each observation job is composed of the following:
0035         </para>
0036         <itemizedlist>
0037             <listitem>
0038                 <para>
0039                     <guilabel>Target name and coordinates</guilabel>: Select target from the <link linkend="findobjects">Find Dialog</link> or <guibutton>Add</guibutton> it from <link linkend="tool-obsplanner">Observation Planner</link>. You can also enter a custom name.
0040                 </para>
0041             </listitem>
0042             <listitem>
0043                 <para>
0044                     Optional <guilabel>FITS File</guilabel>: If a FITS file is specified, the astrometry solver shall solve the file and use the central RA/DEC as the target coordinates.
0045                 </para>
0046             </listitem>
0047             <listitem>
0048                 <para>
0049                     <guilabel>Sequence</guilabel> file: The sequence file is constructed in the Ekos <link linkend="ekos-capture">Capture Module</link>. It contains the number of images to capture, filters, temperature settings, prefixes, download directory, &etc;
0050                 </para>
0051             </listitem>
0052             <listitem>
0053                 <para>
0054                     <guilabel>Priority</guilabel>: Set job priority in the range of 1 to 20 where 1 designates the highest priority and 20 the lowest priority. Priority is applied in calculating the weight used to select the next target to image.
0055                 </para>
0056             </listitem>
0057             <listitem>
0058                 <para>
0059                     <guilabel>Profile</guilabel>: Select which equipment profile to utilize when starting Ekos. If Ekos &amp; INDI are already started and online, this selection is ignored.
0060                 </para>
0061             </listitem>
0062             <listitem>
0063                 <para>
0064                     <guilabel>Steps</guilabel>: The user selects which Ekos modules should be utilized in the observation job execution workflow.
0065                 </para>
0066             </listitem>
0067             <listitem>
0068                 <para>
0069                     <guilabel>Startup Conditions</guilabel>: Conditions that must be met <emphasis role="bold">before</emphasis> the observation job is started. Currently, the user may select to start as soon as possible, <guimenuitem>ASAP</guimenuitem>, or when the target is near or past culmination, or at a specific time.
0070                 </para>
0071             </listitem>
0072             <listitem>
0073                 <para>
0074                     <guilabel>Constraints</guilabel>: Constraints are conditions that must be met <emphasis role="bold">at all times</emphasis> during the observation job execution process. These include minimum target altitude, minimum moon separation, twilight observation, artificial horizon altitude constraints, and weather monitoring.
0075                 </para>
0076             </listitem>
0077             <listitem>
0078                 <para>
0079                     <guilabel>Completion Conditions</guilabel>: Conditions that trigger completion of the observation job. The default selection is to simply mark the observation job as complete once the sequence process is complete. Additional conditions enable the user to repeat the sequence process indefinitely or up until a specific time.
0080                 </para>
0081             </listitem>
0082         </itemizedlist>
0083         <para>
0084             You must select the <guilabel>Target</guilabel> and <guilabel>Sequence</guilabel> before you can add a job to the Scheduler. When the scheduler starts, it evaluates all jobs in accord to the conditions and constraints specified and attempts to select the best job to execute. Selection of the job depends on a simple heuristic algorithm that scores each job given the conditions and constraints, each of which is weighted accordingly. If two targets have identical conditions and constraints, usually the higher priority target followed by higher altitude target is selected for execution. If no candidates are available at the current time, the scheduler goes into sleep mode and wakes up when the next job is ready for execution.
0085         </para>
0086         <screenshot>
0087             <screeninfo>
0088                 Scheduler + Planner
0089             </screeninfo>
0090             <mediaobject>
0091                 <imageobject>
0092                     <imagedata fileref="scheduler_planner.png" format="PNG"/>
0093                 </imageobject>
0094                 <textobject>
0095                     <phrase>Scheduler + Planner</phrase>
0096                 </textobject>
0097             </mediaobject>
0098         </screenshot>
0099         <para>
0100             The description above only tackles the <emphasis role="bold">Data Acquisition</emphasis> stage of the observatory workflow. The overall procedure typically utilized in an observatory can be summarized in three primary stages:
0101         </para>
0102         <orderedlist>
0103             <listitem>
0104                 <para>
0105                     Startup
0106                 </para>
0107             </listitem>
0108             <listitem>
0109                 <para>
0110                     Data Acquisition (including preprocessing and storage)
0111                 </para>
0112             </listitem>
0113             <listitem>
0114                 <para>
0115                     Shutdown
0116                 </para>
0117             </listitem>
0118         </orderedlist>
0119     </sect3>
0120 
0121     <sect3 id="ekos-scheduler-startup-procedure">
0122         <title>Startup Procedure</title>
0123         <para>
0124             Startup procedure is unique to each observatory but may include:
0125         </para>
0126         <itemizedlist>
0127             <listitem>
0128                 <para>
0129                     Turning on power to equipment
0130                 </para>
0131             </listitem>
0132             <listitem>
0133                 <para>
0134                     Running safety/sanity checks
0135                 </para>
0136             </listitem>
0137             <listitem>
0138                 <para>
0139                     Checking weather conditions
0140                 </para>
0141             </listitem>
0142             <listitem>
0143                 <para>
0144                     Turning off light
0145                 </para>
0146             </listitem>
0147             <listitem>
0148                 <para>
0149                     Fan/Light control
0150                 </para>
0151             </listitem>
0152             <listitem>
0153                 <para>
0154                     Unparking dome
0155                 </para>
0156             </listitem>
0157             <listitem>
0158                 <para>
0159                     Unparking mount
0160                 </para>
0161             </listitem>
0162             <listitem>
0163                 <para>
0164                     &etc;
0165                 </para>
0166             </listitem>
0167         </itemizedlist>
0168         <para>
0169             Ekos Scheduler only initiates the startup procedure once the startup time for the first observation job is close (default <emphasis>lead</emphasis> time is 5 minutes before <emphasis>startup</emphasis> time). Once the startup procedure is completed successfully, the scheduler picks the observation job target and starts the sequence process. If a startup script is specified, it shall be executed first.
0170         </para>
0171     </sect3>
0172 
0173     <sect3 id="ekos-scheduler-data-acquisition">
0174         <title>Data Acquisition</title>
0175         <para>
0176             Depending the on the user selection, the typical workflow proceeds as follows:
0177         </para>
0178         <itemizedlist>
0179             <listitem>
0180                 <para>
0181                     Slew mount to target. If a FITS file was specified, it first solves the files and slew to the file coordinates.
0182                 </para>
0183             </listitem>
0184             <listitem>
0185                 <para>
0186                     Auto-focus target. The autofocus process automatically selects the best star in the frame and runs the autofocus algorithm against it.
0187                 </para>
0188             </listitem>
0189             <listitem>
0190                 <para>
0191                     Perform plate solving, sync mount, and slew to target coordinates.
0192                 </para>
0193             </listitem>
0194             <listitem>
0195                 <para>
0196                     Perform post-alignment focusing since the frame might have moved during the plate solving process.
0197                 </para>
0198             </listitem>
0199             <listitem>
0200                 <para>
0201                     Perform calibration and start auto-guiding: The calibration process automatically selects the best guide star, performs calibration, and starts the autoguide process.
0202                 </para>
0203             </listitem>
0204             <listitem>
0205                 <para>
0206                     Load the sequence file in the <link linkend="ekos-capture">Capture module</link> and start the imaging process.
0207                 </para>
0208             </listitem>
0209         </itemizedlist>
0210     </sect3>
0211 
0212     <sect3 id="ekos-scheduler-shutdown">
0213         <title>Shutdown</title>
0214         <para>
0215             Once the observation job is completed successfully, the scheduler selects the next target. If the next target scheduled time is not due yet, the mount is parked until the target is ready. Furthermore, if the next scheduled target is not due for a user-configurable time limit, the scheduler performs a <emphasis>preemptive</emphasis> shutdown to preserve resources and performs the startup procedure again when the target is due.
0216         </para>
0217         <para>
0218             If an unrecoverable error occurs, the observatory initiates shutdown procedure. If there is a shutdown script, it will be executed last.
0219         </para>
0220         <para>
0221             The following video demonstrates an earlier version of the scheduler, but the basic principles still apply today:
0222         </para>
0223         <mediaobject>
0224             <videoobject>
0225                 <videodata contentdepth="315" contentwidth="560" fileref="https://www.youtube.com/embed/v8vIXD1kois"/>
0226             </videoobject>
0227             <caption>
0228                 <para>
0229                     <phrase>Ekos Scheduler</phrase>
0230                 </para>
0231             </caption>
0232         </mediaobject>
0233     </sect3>
0234 
0235     <sect3 id="ekos-scheduler-weather-monitoring">
0236         <title>Weather Monitoring</title>
0237         <para>
0238             Another critical feature of any remotely operated robotic observatory is weather monitoring. For weather updates, Ekos relies on the selected INDI weather driver to continuously monitor the weather conditions. For simplicity sake, the weather conditions can be summed in three states:
0239         </para>
0240         <orderedlist>
0241             <listitem>
0242                 <para>
0243                     <emphasis role="bold">Ok</emphasis>: Weather conditions are clear and optimal for imaging.
0244                 </para>
0245             </listitem>
0246             <listitem>
0247                 <para>
0248                     <emphasis role="bold">Warning</emphasis>: Weather conditions are not clear, seeing is subpar, or partially obstructed and not suitable for imaging. Any further imaging process is suspended until the weather improves. Warning weather status does not pose any danger to the observatory equipment so the observatory is kept operational. The exact behavior to take under Warning status can be configured.
0249                 </para>
0250             </listitem>
0251             <listitem>
0252                 <para>
0253                     <emphasis role="bold">Alert</emphasis>: Weather conditions are detrimental to the observatory safety and shutdown must be initiated as soon as possible.
0254                 </para>
0255             </listitem>
0256         </orderedlist>
0257     </sect3>
0258 
0259     <sect3 id="ekos-scheduler-startup-and-shutdown-scripts">
0260         <title>Startup &amp; Shutdown Scripts</title>
0261         <para>
0262             Due to the uniqueness of each observatory, Ekos enables the user to select startup and shutdown scripts. The scripts take care of any necessary procedures that must take place on startup and shutdown stages. On startup, Ekos executes the startup scripts and only proceeds to the remainder of the startup procedure (unpark dome/unpark mount) if the script completes successfully. Conversely, the shutdown procedure begins with parking the mount &amp; dome before executing the shutdown script as the final procedure.
0263         </para>
0264         <para>
0265             Startup and shutdown scripts can be written any language that can be executed on the local machine. It must return 0 to report success, any other exist value is considered an error indicator. The script's standard output is also directed to Ekos logger window. The following is an sample demo startup script in Python:
0266         </para>
0267         <programlisting language="python">
0268 #!/usr/bin/env python
0269 # -*- coding: utf-8 -*-
0270 
0271 import os
0272 import time
0273 import sys
0274 
0275 print "Turning on observatory equipment..."
0276 sys.stdout.flush()
0277 
0278 time.sleep(5)
0279 
0280 print "Checking safety switches..."
0281 sys.stdout.flush()
0282 
0283 time.sleep(5)
0284 
0285 print "All systems are GO"
0286 sys.stdout.flush()
0287 
0288 exit(0)
0289         </programlisting>
0290         <para>
0291             The startup and shutdown scripts must be <emphasis>executable</emphasis> in order for Ekos to invoke them (&eg; use <userinput>chmod +x startup_script.py</userinput> to mark the script as executable). Ekos Scheduler enables truly simple robotic operation without the need of any human intervention in any step of the process. Without human presence, it becomes increasingly critical to gracefully recover from failures in any stage of the observation run. Using &plasma; notifications, the user can configure audible alarms and email notifications for the various events in the scheduler.
0292         </para>
0293     </sect3>
0294 
0295     <sect3 id="ekos-scheduler-mosaic-Planner">
0296         <title>Mosaic Planner</title>
0297         <screenshot>
0298             <screeninfo>
0299                 Mosaic Planner
0300             </screeninfo>
0301             <mediaobject>
0302                 <imageobject>
0303                     <imagedata fileref="mosaic_planner.png" format="PNG"/>
0304                 </imageobject>
0305                 <textobject>
0306                     <phrase>Mosaic Planner</phrase>
0307                 </textobject>
0308             </mediaobject>
0309         </screenshot>
0310         <para>
0311             Hubble-like super wide field images of <ulink url="http://darkskyart.com/?page_id=96">galaxies</ulink> and nebulae are truly awe-inspiring, and while it takes great skills to obtain such images and process them; many notable names in the field of astrophotography employ gear that is not <emphasis>vastly</emphasis> different from yours or mine. I emphasize <emphasis>vastly</emphasis> because some do indeed have impressive equipment and dedicated observatories worth tens of the thousands of dollars. Nevertheless, many amateurs can obtain stellar wide-field images by combining smaller images into a single grand mosaic.
0312         </para>
0313         <para>
0314             We are often limited by our camera+telescope Field of View (FOV). By increasing FOV by means of a focal reducer or a shorter tube, we gain a larger sky coverage at the expense of spatial resolution. At the same time, many attractive wide-field targets span multiple FOVs across the sky. Without any changes to your astrophotography gear, it is possible to create a super mosaic image <emphasis>stitched</emphasis> together from several smaller images. There are two major steps to accomplish a super mosaic image:
0315         </para>
0316         <orderedlist>
0317             <listitem>
0318                 <para>
0319                     Capture multiple images spanning the target with some overlap between images. The overlap is necessary to enable the processing software from aligning and joining the sub-images.
0320                 </para>
0321             </listitem>
0322             <listitem>
0323                 <para>
0324                     Process the images and <emphasis>stitch</emphasis> them into a super mosaic image.
0325                 </para>
0326             </listitem>
0327         </orderedlist>
0328         <para>
0329             The 2nd step is handled by image processing applications such as <ulink url="https://pixinsight.com">PixInsight</ulink>, among others, and will not be the topic of discussion here. The first step can be accomplished in Ekos Scheduler where it creates a mosaic suitable for your equipment and in accordance with the desired field of view. Not only Ekos creates the mosaic panels for your target, but it also constructs the corresponding observatory jobs required to capture all the images. This greatly facilitates the logistics of capturing many images with different filters and calibration frames across a wide area of the sky.
0330         </para>
0331         <para>
0332             The <guilabel>Mosaic Planner</guilabel> in the Ekos Scheduler will create multiple Scheduler jobs based on a central target. To toggle the planner, click on the <guibutton>Mosaic Planner</guibutton> button in Ekos Scheduler or KStars INDI toolbar as illustrated in the screenshot. The planner draws the Mosaic Panel directly unto the sky map. It is recommended to enable HiPS overlay for the best experience. The planner is composed of four stages:
0333         </para>
0334         <orderedlist>
0335             <listitem>
0336                 <para>
0337                     <emphasis role="bold">Confirm Equipment</emphasis>: Ekos attempts to load equipment settings from INDI. If unsuccessful, you need to enter your equipment settings including your telescope focal length in addition to camera's width, height, and pixel dimensions. The settings are saved for future sessions.
0338                 </para>
0339             <screenshot>
0340             <screeninfo>
0341             Confirm Equipment
0342             </screeninfo>
0343             <mediaobject>
0344                 <imageobject>
0345                     <imagedata fileref="mosaic_confirm_equipment.png" format="PNG"/>
0346                 </imageobject>
0347                 <textobject>
0348                     <phrase>Confirm Equipment</phrase>
0349                 </textobject>
0350             </mediaobject>
0351         </screenshot>
0352             </listitem>
0353             <listitem>
0354                 <para>
0355                     <emphasis role="bold">Adjust Grid</emphasis>: Select the mosaic panel dimension and overlap percentage. The Mosaic Panel is updated accordingly on the sky map. Adjust the Position Angle to match the desired mosaic orientation in the sky. If the Position Angle is different from your camera's usual orientation, you may need to rotate the camera either manually or via a mechanized rotator when the scheduler jobs are executed. Tile transparency is automatically calculated by default but may be turned off and adjusted manually. To compute the mosaic field from the number of tiles, click the <guibutton>Cover FOV</guibutton> button. The mosaic panel can be centered in the sky map by clicking on the <guibutton>Recenter</guibutton> button.
0356                 </para>
0357                 <screenshot>
0358             <screeninfo>
0359             Select Grid
0360             </screeninfo>
0361             <mediaobject>
0362                 <imageobject>
0363                     <imagedata fileref="mosaic_select_grid.png" format="PNG"/>
0364                 </imageobject>
0365                 <textobject>
0366                     <phrase>Select Grid</phrase>
0367                 </textobject>
0368             </mediaobject>
0369         </screenshot>
0370         <para>
0371             A large overlap will make frame stitching easier during post-processing, but it requires more panes to cover the desired extent. However, if you already know the minimal amount of sub-frames your rejection algorithm will use during post-processing, you may want to increase the overlap to attain that amount on the areas covered by multiple panes. For instance, a 4x4 mosaic grid with 75% overlap has 16 sub-frames covering the central intersection, which is enough for Winsorized Sigma rejection. Although the resulting stack does not have the same height on all parts of the final frame, this method gives you control on signal-to-noise ratio and allows you to provide context to your target while exposing a relatively low number of captures.
0372         </para>
0373         <para>
0374             The large number drawn in the corner of each grid pane represents the order in which panes will be captured. The default S-shaped choice (west-east then alternating high-low/low-high moves), ensures minimal movement of the mount during observation. Uncheck <guilabel>Minimum mount move</guilabel> to revert to west-east/high-low movement only. The coordinates of each pane are rendered in their center as degrees, minutes and seconds. Finally, the angle each pane rotates from the center of the mosaic is displayed at the bottom. If your field of view is large, or if your mosaic is located close to a celestial pole, you may observe that rendered panes start rotating visibly due their horizontal position or high declination. Use the <guilabel>overlap</guilabel> to ensure panes cover the desired frame extents properly.
0375         </para>
0376         <screenshot>
0377             <screeninfo>
0378                 Scheduler Mosaic Tool - Large rotation
0379             </screeninfo>
0380             <mediaobject>
0381                 <imageobject>
0382                     <imagedata fileref="mosaic_close_pole.png" format="PNG"/>
0383                 </imageobject>
0384                 <textobject>
0385                     <phrase>Scheduler Mosaic Tool - Large rotation</phrase>
0386                 </textobject>
0387             </mediaobject>
0388         </screenshot>
0389             </listitem>
0390             <listitem>
0391                 <para>
0392                     <emphasis role="bold">Adjust Grid</emphasis>: Adjust Grid center by manually entering the J2000 center or by dragging the center of the mosaic on the sky map.
0393                 </para>
0394             <screenshot>
0395             <screeninfo>
0396             Adjust Grid
0397             </screeninfo>
0398             <mediaobject>
0399                 <imageobject>
0400                     <imagedata fileref="mosaic_adjust_grid.png" format="PNG"/>
0401                 </imageobject>
0402                 <textobject>
0403                     <phrase>Adjust Grid</phrase>
0404                 </textobject>
0405             </mediaobject>
0406         </screenshot>
0407             </listitem>
0408             <listitem>
0409                 <para>
0410                     <emphasis role="bold">Create Jobs</emphasis>: The final step is to select the sequence file and directory to store the images. Target field is automatically filled but may be changed as desired. Select the steps each scheduler job should execute in sequence (Track -> Focus -> Align -> Guide -> Capture), and adjust the frequency of automatic alignment and focus routines that must be executed during the mosaic operation. For example, if <guilabel>Align Every</guilabel> is set to 2 Scheduler Jobs, then the first job will run the astrometry alignment, while the second job will skip it. When the third job is executed, alignment is performed again and so forth.
0411                 </para>
0412             <screenshot>
0413             <screeninfo>
0414             Create Jobs
0415             </screeninfo>
0416             <mediaobject>
0417                 <imageobject>
0418                     <imagedata fileref="mosaic_create_jobs.png" format="PNG"/>
0419                 </imageobject>
0420                 <textobject>
0421                     <phrase>Create Jobs</phrase>
0422                 </textobject>
0423             </mediaobject>
0424         </screenshot>
0425             </listitem>
0426         </orderedlist>                
0427         <para>
0428             Click <guibutton>Create Jobs</guibutton> to generate mosaic scheduler jobs and add them to the schedule queue.
0429         </para>
0430     </sect3>
0431 </sect2>