More

GP service (from Python script tool) time-out: how to execute finally: statement


I am trying to build more fail-management logic into some of the GP services I develop. As a part of the business logic within the script tool I build, I write some data into the tables and thus lock access to certain parts of the geodatabase to let the currently running GP task to be executed undisturbed.

However, in case there are some issues which will force the GP service instance or ArcGIS Server Windows service to crash (something that is out of my hands), I want to be able to perform some clean-up work (release locks, clean-up scratch geodatabases, write to tables, etc). This is where I usually usefinally:statement when developing Python script tools for use in ArcGIS Desktop.

However, the GP service does not seem to reach thefinally:if it is timed-out or crashed. The Python script tool source code is below:

import arcpy import time try: arcpy.AddMessage("Hello 1 second") time.sleep(10) arcpy.AddMessage("Hello after 10 sec") except: pass finally: arcpy.AddMessage("GP service timed out!")

I have run this tool in ArcMap:

Trying to run the GP service task (published from the tool result) that of GP service with time-out time for 9 seconds:

So the GP service is not able to reachfinallystatement and print"Hello after 10 sec". If this is not possible to implement within thefinally:statement in Python script tool source code, how would you approach this otherwise?


What you're asking for isn't possible. ArcGIS server supervises the Python process and kills it if it takes too long. It doesn't send an interrupt/break to the Python session so the Python interpreter would never get to thatfinallyblock. I know you said it's out of your control, but you may want to beg the administrators of your server to let you tick up the timeout of the service to a larger value.


Domoticz

This system is designed to operate in various operating systems. The user-interface is a scalable HTML5 web frontend, and is automatically adapted for Desktop and Mobile Devices.
Compatible with all browsers. Note for Internet explorer users: You need version 10+.

Some Information

  • Hardware: RFXCOM Transceiver, Z-Wave, P1 Smart Meter, YouLess Meter, Pulse Counters, 1-Wire, EnOcean, and a lot more.
  • Extended logging
  • iPhone / Android push notifications
  • Auto Learning sensors/switches
  • Manual creation of switch codes
  • Share / Use external devices
  • Designed for simplicity

Donations

We use Matomo Analytics for website analytics which respects your privacy!


Some rules of thumb for subprocess .

  • Never use shell=True . It needlessly invokes an extra shell process to call your program.
  • When calling processes, arguments are passed around as lists. sys.argv in python is a list, and so is argv in C. So you pass a list to Popen to call subprocesses, not a string.
  • Don't redirect stderr to a PIPE when you're not reading it.
  • Don't redirect stdin when you're not writing to it.

That said, it is probable that rsync buffers its output when it detects that it is connected to a pipe instead of a terminal. This is the default behavior - when connected to a pipe, programs must explicitly flush stdout for realtime results, otherwise standard C library will buffer.

To test for that, try running this instead:

and create a test_out.py file with the contents:

Executing that subprocess should give you "Hello" and wait 10 seconds before giving "World". If that happens with the python code above and not with rsync , that means rsync itself is buffering output, so you are out of luck.

A solution would be to connect direct to a pty , using something like pexpect .


Developing a web-based system for supervised classification of remote sensing images

Web-based image classification systems aim to provide users with an easy access to image classification function. The existing work mainly focuses on web-based unsupervised classification systems. This paper proposes a web-based supervised classification system framework which includes three modules: client, servlet and service. It comprehensively describes how to combine the procedures of supervised classification into the development of a web system. A series of methods are presented to realize the modules respectively. A prototype system of the framework is also implemented and a number of remote sensing (RS) images are tested on it. Experiment results show that the prototype is capable of accomplishing supervised classification of RS images on the Web. If appropriate algorithms and parameter values are used, the results of the web-based solution could be as accurate as the results of traditional desktop-based systems. This paper lays the foundation on both theoretical and practical aspects for the future development of operational web-based supervised classification systems.

This is a preview of subscription content, access via your institution.


[Udemy 100% Off]-Applied Ethical Hacking and Rules of Engagement

Get 100% Free Udemy Discount Coupon Code ( UDEMY Free Promo Code ) ,You Will Be Able To Enroll this Course “Applied Ethical Hacking and Rules of Engagement” totally FREE For Lifetime Access . Do Hurry Or You Will Have To Pay $ $

Requirements

  • No prior Linux, programming or hacking knowledge required
  • A Computer with Internet connection
  • Passionate curiosity for learning (is a must)
  • Basic IT Skills (is plus to have)

Course Duration: 40 hours

Instructor: Seyed Farshid Miri

English Ratings: 4.6

Description

<<< Welcome to the most complete Ethical Hacking and Threat Hunting course available online, where both topics in offensive security, as well as defensive security, are professionally covered. This course includes two crash courses about Linux and Python as well.>>>

The following seven pillars constitute the foundation of this life-changing course:

######################################################################
1- Ethical Hacking
Learn how to think and act like a hacker and work with various techniques and tools to achieve this goal. As an ethical hacker at the end of this course, you will be able to help your customers mitigate various attack vectors and their corresponding details practically based on various security standards and best practices. Also, you will learn how to execute various ethical hacking phases as Reconnaissance, Scanning, Gaining Access, Maintaining Access, Clearing Tracks, and others.

######################################################################
2- Penetration Testing
Learn how to hack easy to hard real-world simulated virtual machines on HackTheBoxLive Hacking! using unique exploits, tactics, and techniques. Learn the art of intrusion with these CTFs (Capture the Flags) which will help you in the future on every real work project.
Also work on pentest methods in web, network, vulnerability assessment workflows, and “Defense in Depth” best practices which will help you hack like black-hat hackers, defend or secure them like security experts and harden your corporate environment against malicious actors.

Learn beginner to advanced pentesting techniques. Learn how to think and act like threat actors to stop them at various phases of the attack life cycle.
MITRE ATT&CK Framework: reconnaissance, initial foothold, lateral movement, privilege escalation, command and control, active directory attacks, Linux, and mac os x malware and attack techniques.
Learn scripting languages for the Cobalt Strike Framework and other red-team engagement frameworks to perform development and operations on them.
Learn how to develop your C2 infrastructure to avoid detection by blue teams and SOCs during red team operations.

######################################################################
4- Elastic Stack Wazuh Manager (SIEM)
Learn how to set up a complete SIEM (Security Information and Event Management) using Elastic Stack (formerly ELK Stack) using Wazuh Manager. Also, learn how to ingest various log formats from different log sources such as Linux and Windows servers, Fortigate firewall appliances, and so on. You will learn how to activate different functionalities (capabilities) of the Wazuh manager such as vulnerability monitoring, File Integrity Monitoring, CIS Hardening Benchmark Monitoring, and much more. Also, you will learn how the underlying decoders and rules are programmed to detect an unlimited amount of security events across an enterprise network.

######################################################################
5- Threat Hunting (Blue-Teaming)
There is a complete section for threat hunting where you put what you’ve learned into work and run attacks such as Spawn Session and Process Injection, ShellShock, MSHTA, Brute-Force, Mimikatz, and so on from your Parrot OS and detect them with your SIEM tool that you’ve set up and completely configured during the course. During this section, you get familiar with how different IoC (Indication of Compromise) will appear in your SIEM tool.

######################################################################
6- Python Scripting for Security
Learn how to create scripts and programs to do what you want whenever you are required to, from small scripts that are needed during pentest to more sophisticated ones during Red Team Ops. there is a crash course about Python basics included in this course to promote you in this must-know language field.

######################################################################
7- Linux (Kali Linux and Parrot OS)
Linux runs the world, especially when it comes to the cybersecurity world. There is a crash course about Linux basics in this course. However, during this course and after many hours of exciting hands-on practices on the different offensive and defensive security methods you will become a Linux expert at the level of a cybersecurity expert. You will learn Kali Linux and Parrot OS as the main Linux distros used in this course.


Street data processing toolbox from Esri

Recently I was doing quite a lot of work with street data processing so that one could get network dataset in ArcGIS based on raw data coming as Navstreets shapefiles from HERE (known also as Nokia and previously Navteq).

Just for some years ago, there was no other way to compile those shapefiles into network dataset except from doing a lot of operations manually and use ArcObjects/Python. Fortunately, Esri has published a Street data processing toolbox which would do just that. It is free and supports both Navteq Navstreets data as well as TomTom Multinet.

I was worrying a bit first because I was not sure whether the SDP toolbox is capable of compiling really large datasets covering a bigger part of Europe and North America. However, it went really well and huge datasets were compiled within a week. I have been using some machines I had at my disposal, but I have run out of RAM quite soon. I had to use Amazon EC2 running a rather powerful instance. As a best practice, in addition to running on a 64-bit OS and having lots of RAM, you should also be sure that you have plenty of temporary disk space on the system drive (the temp folder is used heavily for the temp data storage). Note that the temporary space is not the same as the destination for the output file geodatabase where the network dataset will be created, and is often located at C:Users\%username%AppDataLocalTemp. The USA+Canada network dataset was compiled in 70 hours (Amazon EC2 m2.2xlarge instance was used).

Be aware that SDP tools cannot be run with background processing enabled, so you should author and run a tool via a model or Python script tool if you want to run some custom data processing operations first. Remember to switch off the background processing in ArcGIS Desktop application (Geoprocessing > Geoprocessing Options). You can run Dissolve Network GP tool to get a more compiled network dataset which will be faster in operation (remember to build a network dataset after dissolving) and then compress file geodatabase with Compress Geodatabase GP tool to make it smaller, perform faster and be read-only. The network dataset in this format (dissolved network and compressed geodatabase) performs as good as SDC.

Rate this:


About These BI Tools

Whether you’re just starting or in the middle of your search, you may have already heard of Microsoft Power BI, Tableau and QlikView, as they are all three quite popular BI solutions. However, you probably want to know how they stack up against each other and more importantly, against your BI requirements. We’ll briefly take a look at an overview of their features and what makes them unique.

Power BI

Microsoft Power BI is an analytics and data visualization tool that enables users to connect to a wide variety of data sources and create reports and dashboards. Though it originally emerged as an add-on to Microsoft Excel, Power BI has since grown to stand on its own as a formidable product that fulfills the needs of all companies, from small businesses to large enterprises. Its highlights include seamless integration with Microsoft Office technologies, powerful data preparation and data querying capabilities, a drag-and-drop data visualization builder and a perpetually free version.

Tableau

Tableau is a data visualization solution that excels in helping businesses turn their data into actionable insights through interactive dashboards and stories. With a high degree of customization and security options, it offers users considerable control over their data storytelling. Through a user-friendly interface and intuitive drag-and-drop functionalities, it encourages users of all technical skill levels to create and explore data visualizations. While Tableau Desktop is the main offering, it comes as a part of several different packages that pair it with a license for Tableau Server or Tableau Online.

Qlikview

QlikView is a data discovery and analytics platform from Qlik Technologies that helps companies make data-driven decisions by making insights accessible to business users. Built for speedy deployment and quick configuration, it connects directly to and pulls information from almost any data source and then allows users to analyze all their data and build visualizations that convey its story effectively. Its differentiators include the Qlik Associative Engine, which indexes every possible relationship among data to render faster insights and enable associative analysis, a data governance dashboard and wizards for alerts, reporting and more. Though the vendor encourages users to move to its cloud-based QlikSense platform, it continues to provide updates and support for QlikView, as it remains a strong competitor in the business intelligence field.

This is just a brief summary of these three BI leaders and their features. For even more details, make sure to check out their product pages and request our free pricing guide.


Related:

  1. Read more about bc and printf in my answer here: https://stackoverflow.com/questions/12722095/how-do-i-use-floating-point-division-in-bash/58479867#58479867
  2. I don't remember where I first learned about the time command anymore, but it may have been from @Trudbert's answer right here.

Using only bash it is also possible to measure and calculate the time duration for a portion of the shell script (or the elapsed time for the entire script):

you can now either just print the difference:

if you only need incremental duration do:

You can also store the duration in a variable:

Timing function based on SECONDS , limited to second-level granularity only, doesn't use any external commands:

The accepted solution using time writes to stderr .
The solution using times writes to stdout .
The solutions using $SECONDS are missing sub-second precision.
The other solutions involve calling external programs like date or perf which is not efficient.
If you are fine with any of these, use it.

But if you need an efficient solution to get the times with millisecond precision and need them into variables such that the original output remains undisturbed you may combine process substitution with some redirections around time which is much faster than calling external programs and allows redirections around the timing wrapper script as on the original command/script.

Select one of the following variants parsing the output of time appropriate to you needs:
Shortest variant where stdout of $Cmd is written to stderr and nothing to stdout :

Longer variant that keeps original stdout and stderr separate of each other:

Most complicated variant includes closing the extra file descriptors such that $Cmd is called as if without this timing wrapper around it and lvm commands like vgs do not complain about leaked file descriptors:

You can even fake a floating point addition in bash without calling bc which would be much slower:


Presentation of results

After benchmarking results have been obtained, it is typically necessary to present them in a way that is both scientifically valid as well as comprehensible for humans. In this section, we first remind the reader of basic, but important formal aspects, which need to be respected when presenting measurement results. Following this, we discuss different methods that are well-suited for comparing two or more tools.

Note that all visualizations of benchmark results in this paper (the plots from Sect. 4 as well as those in this section) have been automatically generated from BenchExec result files using the LaTeX package pgfplots . Footnote 33 Documentation on how to achieve this is available in the BenchExec repository. Footnote 34

Required information

In order to make experiments replicable, some information needs to be given about the benchmarking setup. While it is in general infeasible to provide a complete specification of the benchmarking machine (e.g., including versions of every package on the system), Footnote 35 the information that is expected to have the most significant potential impact on benchmarking results needs to be given. We typically consider this to include

CPU model and size of RAM,

specified resource limits,

version of important software packages, such as the kernel or runtime environments like the Java VM,

version and configuration of the benchmarked tool(s), and

version of the benchmark set (input files).

For replicability it is further a requirement that the benchmarked tool(s) and the input files are made available, e.g., on a supplementary web page, as additional material in the digital library of the publisher, Footnote 36 or in designated online repositories. Footnote 37 We recommend to also make the raw benchmarking results and used data-evaluation tools and scripts available in order to make not only the experiment but also the data analysis replicable.

Units

In general, it is recommended to use SI units and prefixes wherever possible. In particular, special care needs to be taken for values measured in bytes, such as memory or file size. The prefixes kilo (k), mega (M), giga (G), etc. are defined as decimal prefixes, i.e., they always represent a factor of 1000 compared to the next smaller prefix. For example, one megabyte (MB) is exactly 1 000 000 bytes. It is incorrect to use these prefixes as representation for a factor of 1024. While this mistake would create an error of already 2.4% for the prefix kilo, the error would increase to 4.9% for values with the prefix mega, and to 7.4% for values with the prefix giga, which would typically be regarded as a significant systematic error.

Using the common decimal prefixes kilo, mega, etc. is preferred. In cases where the binary prefixes kibi, mebi, gibi, etc. with factors of 1024 are more common (such as for amounts of memory), it is also possible to use these prefixes, if the correct symbols KiB, MiB, GiB, etc. are used. Note that for the prefix kilo a lowercase letter is used as symbol, while for all other prefixes that represent an integral factor (including kibi) an uppercase letter is used.

Significant digits

When writing about measurement results or presenting an overview thereof in tables, it is important to define the number of significant digits to be used throughout the presentation. In this context, it is of importance whether or not a data item being presented is prone to errors in measurement or not. For example, one can provide exact numbers for the number of tasks solved by a tool, or for the number of lines of an input file. Hence, these numbers can be given with an arbitrary, but fixed number of digits. With exact numbers, it is also acceptable to round data items to allow a more compact presentation. For example, the length of an input program measured in lines of code (LOC) could be given rounded to 1000 lines with the unit kLOC.

For inexact numbers and measures, for example obtained by measuring time or memory consumption, this is different. One has to define how precise a measurement is, and show all results with a constant number of significant digits according to the precision of the measurement. For example, the maximal error of a measurement might be 1, or 0.1, or 0.01%, which would lead to two, three, and four significant digits, respectively. As a concrete example, consider a time measurement with an assumed precision of 0.01%. If the value 43.2109 s is measured, this needs to be reported with four significant digits as 43.21 s, whereas the value 432.1098 s would be shown as 432.1 s. Large and small numbers (e.g., 43210 and 0.04321 s) need to be padded with (insignificant) zero digits. Because from the number 43210 s alone it is not clear whether it has 4 or 5 significant digits, the number of significant digits needs to be explicitly stated for each kind of measurement. This can be avoided by using scientific notation (e.g., (4.321 imes 10^<4>) s), but this might be harder to interpret for human readers.

Note that the number of significant digits is independent from the used unit, and the reported values always contain exactly the same amount of information no matter in which unit they are.

The naive approach of rounding all numbers to a fixed amount of decimal places (e.g., 4.321 and 0.043 s) should not be used. First, for large numbers, this adds too many digits with irrelevant noise and may lead the reader to wrong conclusions about the precision of the measurement. Second, for small numbers, this leads to a large rounding error that can be orders of magnitudes larger than the measurement error.

Comparison of summarized results: scores and rankings

It is often necessary to compare the achievements of two or more different tools or tool configurations in a summarized quantitative way. The way this can be done depends on circumstances such as the research area and the goal of the experimental evaluation. In the following, we discuss summarizing results for cases where the result of a tool for an input file can be categorized as either correct, wrong, or inconclusive (as is common in computer-aided verification), i.e., we ignore cases with quantitative tool results (e.g., the quality of an approximation). Inconclusive results are cases where the tool gave up, crashed, or exceeded its resource limits.

In some communities, a wrong result is considered not acceptable, for example in SAT-COMP [1]. In this case, the tools with wrong results are excluded and tools without wrong results can be compared by their number of correct results. A slightly less strict alternative is to rank tools by comparing first the number of wrong results, and in case of ties the number of correct results. This is for example used by SMT-COMP since 2012 [13].

Sometimes it is desired to define a single measure that summarizes the number of correct and wrong results without penalizing tools with wrong results too strongly, for example, because all available tools potentially might produce some wrong results. A scoring scheme can be used to achieve this and calculate a single score for each tool. A typical scoring scheme is as follows: positive points are given for each correct result, zero points are given for each inconclusive result, and negative points are given for each wrong result [3]. The amount of negative points per wrong result is larger than the amount of positive points per correct result in order to penalize wrong results. For example, in SV-COMP’17 one or two points are given for each correct result and −16 or −32 points are given for each wrong result. Note that in this community, some tasks or results (safety proofs) are considered more difficult than others and lead to a higher number of points, and some forms of incorrect results (unsound results) are considered worse than others and lead to a higher penalty.

Because the definition of the scoring scheme heavily influences the ranking and presentation of the evaluation results, a scoring scheme should be used only if it is generally accepted by the community, for example by adoption in a competition in this area. It is important to always also list the raw numbers of correct and incorrect results (possibly further categorized into unsound and incomplete results, for example) in addition to the score.

Tables

Presenting and comparing numeric data using tables is a straight-forward approach. Authors should respect the suggestions made above regarding the selection of units and stick to a fixed number of significant digits. The latter often results in situations where different cells of the same column have a different number of decimal places, which at first might be counter-intuitive for a reader. If, however, the data in the columns are aligned to the decimal point, the presentation becomes intuitive at a glance, because the larger a value the more it grows to the left, while the smaller a value the more it shifts to the right. Using scientific notation would also be a possibility, but again, comparing along a row or a column is less intuitive, because all entries would have equal length. Table 1 presents a comparison of the various formats. The first column labeled “fixed decimal places” contains all numbers rounded to the same number of decimal places. As described above, this is incorrect due to rounding errors and measurement noise. The remaining columns all show correct ways of presenting measurement values (with four significant digits), but vary according to their readability and specifically with regard to the possibility of comparing the magnitude of the numbers visually. Right-aligned numbers and scientific notation can make it difficult to distinguish between values such as 123.5 and 12.35, which appear similarly but differ by an order of magnitude. For numbers aligned on the decimal separator (as in the last column), the order of magnitude of each value corresponds to its horizontal position within its cell, which allows the reader to get a quick visual overview of the results. Note that we drop the leading zero for values smaller than 1 such that this effect is increased. The recommended rounding and alignment can be produced in LaTeX tables automatically with packages such as siunitx Footnote 38 and pgfplotstable . Footnote 39 The latter even supports creating correctly formatted tables directly from raw CSV files in one step.

Of course, due to space issues, tables can only be used for a small number of results. For larger numbers of results, such as for hundreds or thousands of input files, only summarized data can be shown, and the full set of results should be made available in some other form, e.g., on a supplementary web page or as additional material in a digital library. In addition to showing summarized data, a further possibility is to also show a selection of the most interesting results, such as the n best and worst results Footnote 40 , and to use plots to visualize a large number of results. The benefit of plots is that for humans this form of presentation usually provides a quick understanding of the data. The downside is that small differences between results are harder or impossible to identify. However, if comparing a set of hundreds or thousands of runs, such detail needs to be sacrificed and abstracted anyway. Moreover, a plot, quite literally, scales better than a table, and is therefore more suitable for comparisons of large data sets.

Scatter plots

For comparing two columns of related results, for example time results for two different tools for a common set of input files, scatter plots can be used to give a good overview. Scatter plots are drawn onto a two-dimensional coordinate system with the two axes representing the two result columns (typically of the same unit and range). For each input file, there is one data point with the result of the first tool or configuration as its x-value and the result of the other tool or configuration as its y-value. An example scatter plot with CPU-time results for two different tools can be seen in Fig. 8. We highlighted the data point at (16, 1.1), whose meaning is that there exists an input file for which Tool 1 needed 16 s of CPU time and Tool 2 needed 1.1 s.

Example scatter plot, data taken from two tools of category “DeviceDrivers64” of SV-COMP’15 [5]

In a scatter plot, data points on the diagonal show cases where both tools have the same value, data points in the lower-right half of the plot (below the diagonal) show cases where the value of the tool shown on the y-axis is lower, and data points in the upper-left half of the plot (above the diagonal) show cases where the value of the tool shown on the x-axis is lower, and the distance of a data point from the diagonal correlates to the difference between the results. This allows the reader to easily get a quick overview of the results by looking at the distribution of the data points across the two halves. To further aid this, the diagonal and further parallel lines (for example, the functions (y=x+10) and (y=x-10) , or (y = 10x) and (y = frac<10>) on a logarithmic plot) can be added to the plot.

Compared to a table, scatter plots have the disadvantage that it is not possible to see which input file each data point represents. It is also not possible to show results for input files for which one of the results is missing. Data points where at least one component corresponds to a wrong or inconclusive answer also need to be omitted. Otherwise, it could happen that in a scatter plot of time results a tool with many fast but wrong results or fast crashes would visually appear “better” than another tool with slower but correct results, which should be avoided. An exception to this rule are timeout results in scatter plots of time results, which should be shown with a time equal to the time limit and lead (if present in significant numbers) to characteristic clusters at the top or right of the plot. Thus, scatter plots are especially suited for cases where the number of wrong or inconclusive results (except timeouts) is not significant. In other cases it must be clear that the plot shows only a subset of the results and that this can lead to a skewed picture (e.g., if one tool is particularly good for input files for which the other tool crashes, these data points are missing).

Scatter plots support the comparison of only two result columns. More information can sometimes be added by coloring each data point accordingly. However, care should be taken to ensure that crucial information is still available in black-and-white printouts. One use case for a colored scatter plot is to indicate how many results each data point represents, if for example the axes are discrete and there would be cases with many data points at the same coordinates. An example for such a plot can be seen in Fig. 9. For comparing more results, either several scatter plots or different forms of plots should be used.

Quantile plots

Quantile plots (also known as cactus plots) are plots where the data points represent which quantile (x-value) of the runs need less than the given measure (y-value). Technically, each graph is produced by sorting the data points according to their y-value. This leads to a plot where each graph represents the performance of one tool and the graph is monotonically increasing. Only data points for correct results are shown, and quantile plots are typically used to present time results. Figure 10 shows an example quantile plot with the results of eight tools for 46 input files. The highlighted data point at (21, 5.4) means that Tool 5 produced 21 correct results in at most 5.4 s each and took longer than 5.4 s for its remaining results.

Example scatter plot with color of data points indicating the number of runs each data point represents

In a quantile plot, the right-most x-value of each graph corresponds to the number of correct results that the respective tool produced. The area under each graph roughly indicates the sum of the run times for all correct results of the tool. Thus, in general, a tool could be considered “better” the further to the right its graph stretches and the lower it is. Furthermore, the slope of a graph may indicate how the tool scales for more difficult tasks. Sometimes, the shape of a graph also gives some further indications. For example, the fact that a tool employs a sequential combination of different strategies and switches strategies after some internal time limit is often recognizable by a characteristic kink in the graph at that time limit. The y-value of the left-most data point of a graph often shows the startup time of the tool. Tools that use a virtual machine, e.g., tools programmed in Java, typically have a slight offset here.

Compared to a scatter plot, quantile plots have the advantage that they can visualize an arbitrary number of result columns, and that data points for all correct results of each tool can be shown (and not only for the intersection of correct results), which means that it also shows the effectiveness of each tool and not only its (partial) efficiency. The disadvantage, however, is that there is no relation between the different graphs and no information regarding how the tools compare on a specific input file. In fact, with a quantile plot it could well happen that the graphs of two tools are similar or identical, but there is not a single input file for which the two tools produced a similar result.

Example quantile plot, data taken from category “Simple“ of SV-COMP’15 [5]

Variants of quantile plots

While quantile plots show the number of correct results, they still lack the number of incorrect results. A tool with many correct and incorrect results could misleadingly appear “better” than a tool with a lower number of correct results and no incorrect results. This can be avoided by shifting the graph of each tool to the left by the amount of incorrect results. Now the number of incorrect results is visible in the plot as the negative x-offset of the graph, the number of correct results is visible as the x-length of the graph, and their difference is visible as the x-value of the right-most data point. However, this gives the visual indication that an incorrect result and a correct result are somehow of similar importance and cancel each other out, which may not be desired.

Example score-based quantile plot, data taken from category “Simple“ of SV-COMP’15 [5]

This problem can be avoided with a score-based quantile plot as introduced by SV-COMP’13 [4], if a scoring scheme for summarized results is available (cf. Sect. 7.4). In such a plot, a data point (x, y) means that the sum of the scores for all incorrect results and the sum of the scores of all correct results that were faster than y seconds is x. Thus, for example, a correct result with score 2 makes the graph stretch further to the right (and appear “better”) than a correct result with the same time but score 1.

In a score-based quantile plot, the accumulated score for all incorrect results is visible as the graph’s left-most x-value, the accumulated score for all correct results is visible as the x-length of the graph, and the total score for all results is visible as the x-value of the right-most data point. The general intuition that a graph that is lower and further to the right is “better” still holds. An ideal tool with no wrong results would have its graph start with a positive x-offset.

As example, Fig. 11 shows a score-based quantile plot for the same results as Fig. 10. The highlighted data point corresponds to the same result in both figures, but in Fig. 11 it is drawn at (25, 5.4) because, for the 21 correct results produced by Tool 5 in at most 5.4 s, there were 25 points awarded (and the tool did not produce any wrong answers). This shows how the relative weighting of the results affects the graphs. The influence of incorrect results can be seen, for example, at the graphs of Tool 2, Tool 4, and Tool 6, especially when compared with Tool 3 (cf. the right end of their graphs).


30.9. File Transfer Protocol (FTP)

The File Transfer Protocol (FTP) provides users with a simple way to transfer files to and from an FTP server. FreeBSD includes FTP server software, ftpd, in the base system.

FreeBSD provides several configuration files for controlling access to the FTP server. This section summarizes these files. Refer to ftpd(8) for more details about the built-in FTP server.

30.9.1. Configuration

The most important configuration step is deciding which accounts will be allowed access to the FTP server. A FreeBSD system has a number of system accounts which should not be allowed FTP access. The list of users disallowed any FTP access can be found in /etc/ftpusers . By default, it includes system accounts. Additional users that should not be allowed access to FTP can be added.

In some cases it may be desirable to restrict the access of some users without preventing them completely from using FTP. This can be accomplished be creating /etc/ftpchroot as described in ftpchroot(5). This file lists users and groups subject to FTP access restrictions.

To enable anonymous FTP access to the server, create a user named ftp on the FreeBSD system. Users will then be able to log on to the FTP server with a username of ftp or anonymous . When prompted for the password, any input will be accepted, but by convention, an email address should be used as the password. The FTP server will call chroot(2) when an anonymous user logs in, to restrict access to only the home directory of the ftp user.

There are two text files that can be created to specify welcome messages to be displayed to FTP clients. The contents of /etc/ftpwelcome will be displayed to users before they reach the login prompt. After a successful login, the contents of /etc/ftpmotd will be displayed. Note that the path to this file is relative to the login environment, so the contents of


API design changes¶

The following are some of the high level changes you would notice in the version 1.0 of the API. You might have to update your notebooks and scripts and we apologize for this inconvenience.

  • Various layer types in arcgis.lyr module are now in their own separate modules such as arcgis.features , arcgis.raster , arcgis.network etc.
  • arcgis.viz is now arcgis.mapping with additional support for vector tiles, map image layers
  • Tools in arcgis.tools are accessible as functions in the corresponding layer modules. This allows a better grouping and presentation of tools making them available right alongside the layer types they process. For instance, the feature analysis tools available earlier as arcgis.tools.FeatureAnalysisTools is now arcgis.features.analyze_patterns , arcgis.features.enrich_data etc.
  • Big data analysis tools available earlier as arcgis.tools.BigData is now in a separate module arcgis.geoanalytics .


Watch the video: how to execute a python script every Monday or every day (October 2021).