ICM Scripting Guide

When creating non-interactive scripts in the ICM scripting language, you must use the correct program loader directive. This must reflect a path to your ICM executable. It may be reasonable if the program is to run autonomously to call ICM without the overhead of a graphical user interface. This is done with icmng. The following is a typical useful first line for ICM script files:

The -s option stands for "silent" because it suppresses all of the start up messages.

To find out what version of ICM you are using you can use the following features:

If you’re in an environment where the ICM is updated frequently, it is often useful to consider the date in the version string as this is when this ICM executable was actually compiled.

When ICM starts, it loads a lot of files. These include system-wide and user-specific start-up files, as well as special files in the ICM home directory (like _macro and icm.*).

Moreover, ICM is controlled by preferences. Users may have non-default preferences that are stored and loaded from their home directory. Adding to the complexity is the fact that specific paths and the exact list of loaded files depends on (1) the ICM command line options, (2) the version of ICM, and (3) the operation system.

On Linux, the following locations can contain special user-specific functionality.

When using ICM’s command line interactively, you will see icm/def>. This prompt is defined with the s_icmPrompt variable. It can be set with things like %t> which will show the elapsed time since this shell was started, like this:

What does def in icm/def> mean? Not really sure.

When an object is specified by itself in the shell, the interpreter evaluates and prints object’s value (like the Python interpreter). This can be handy for debugging.

The ICM scripting language usually does not need to decorate variables to indicate that is what they are (like Bash and Perl do with $). It adds defined variables to the correct name space and evaluates them when the name is encountered. However, ICM also allows for explicit variable evaluation. For example:

Don’t get too carried away with this or you’ll get an error like:

Semi-colons can be used to separate commands (since version 3.6-1). They are not required to terminate a statement. Note that the interpreter’s history mechanism will store multiple statements separated by semi-colons as multiple history entries. This differs from other interpreted shells such as Bash or Python. It probably is wise to avoid using semi-colons.

Space is an important delimiter. If you do something like this:

Although space is important here, it’s not as important as it might seem. For example:

In general you can put as many spaces as you like as a delimiter.

At the beginning of a line, a # character causes the interpreter to ignore the following comment. In the middle of a line the # causes the interpreter to ignore subsequent text allowing for comments like:

Interestingly in ICM’s shell history a comment is not even recorded, so making notes to yourself in interactive sessions for history recall probably won’t work.

ICM commands tend to use all lower case letters (example delete). ICM’s built in functions tend to capitalize the first letter of the function (example Nof()). ICM macros tend to use what Perl people call "camel case" where the name is descriptively several concatenated words with each capitalized (example MakePdbFromStereo).

For simple object usage, things are simple:

Assign an integer: a=1

Assign a real: b=1.

Assign a character (string) type: c="a string"

ICM’s scripting language is pretty strongly typed for a scripting language. In Perl or Python a variable can contain one type and then be reassigned to a different type implicitly casting it to that new type. ICM does not force you to explicitly declare a variable’s type but it does require that you be consistent with it as the following example shows:

It is possible to convert variables in place if there is sufficient information. For example:

In this case it was possible to make the 1 into a 1.0 but ICM didn’t want to make the 2.1 into a 2 automatically.

If you want to see all of the integers that are defined, you can use show integer. This works for string, real, map, and others.

One property of ICM that is somewhat idiosyncratic is that many functions, in addition to possibly handling input and output, also set internal ICM shell variables which can be used in subsequent operations. This is a bit like the Perl variable $_ which is magically set after certain functions run, allowing subsequent functions to assume some preparatory action (or as the Perl people say, "underline is understood to be underlying certain undertakings").

In ICM the following automatic variables are sometimes set by functions and can be used by subsequent operations.

Apparently you can modify these variables with some caveats. You can not delete them. Also functions will overwrite anything you put in there if given a chance.

Strings can be defined in a few ways. You can use quotes pretty intuitively:

You can also use a Python style triple quote style like this:

ICM has escape sequences for several common symbols like \", \t, \n.

Strings can be treated like arrays to an extent. You can do something like this similar to Python slices:

Strings can be formatted the C way with the printf command. To just format the string without printing it per se, see sprintf. To send formatted string output to a file, see fprintf (also see the IO section below).

Arrays are ordered containers of consistently typed objects.

The command list is used to "display" variables, i.e. not for anything related to an itemized list in the Python or Lisp sense.

With no arguments, it will give you a list of defined variables.

ICM has a type of array that points to other types of objects. This is a "pointer array", the parray type. This allows you to store heterogeneous typed objects in one object.

The first thing to clarify is that there exist "predefined icm-shell tables" (described here) which are not "user-defined ICM tables" (described here). Just be aware that sometimes tables are referred to (the "Tables" link on the molsoft.com/man page) and they don’t mean the kind you’ll typically use. That said, many ICM features like to generate a table object that the user can use.

Tables are not simply two dimensional arrays. Tables are a first class data type that contain and organize arrays of other types. A table is a specific data structure that may have zero or more columns (arrays with data) and zero or more headers (which also may be arrays).

You can check to see if an ICB file contains a table and what that table’s name is with this.

The most basic way to create a table is to use group table. This takes at least one argument which comes next and must be the name of the table.

To verify that you have created a table or to find your table after forgetting what you named it, use list tables. You’ll notice that there are many built in tables in ICM.

The next thing to do is to add "columns" to the table. This can be with the group table command.

Columns can also be added with the add column command. This also can create empty tables. Here’s an example of both.

If working with tables interactively, you can invoke the system’s text editor and make arbitrary edits to tables with the following command: edit t

You can also change the value of individual cells with a command like:

FAQ: What’s a "map"? A hash like C+\+? An electron density map? It looks like there is a type collection. There is documentation on this and on my ICM they seem to work, but there are also references which make it seem like "collections" are being renamed into "hash" objects. However ER says that "collection" is official and correct and "hash" was some intermediate version.

To use them do something like the following:

FAQ: How do you iterate over the keys in a collection? Answer:

How to tell if an object is a key in a collection (Basically obj.has_key(key) in Python.)?

Apparently Collection(web) is a facility for parsing CGI input but I am still trying to figure out how. Eugene says, "It checks REQUEST_METHOD first, then either parses QUERY_STRING for GET or stdin for POST."

To use collections as function parameters, you need to name the parameter something like: P_MyCollection.

When scripting with ICM it is sometimes desirable to reset the values of all the objects (variables, molecular objects, tables, etc) without having to exit or restart the session. The delete all command will remove everything you created except for protected items (set property delete off/on). This is extremely useful in processing multiple ICB files, a task that ICM scripting frequently performs.

+= Assign result of right hand operator and left hand value to left hand operator. a+=3 is the same as a=a+3. The same is true for subtraction -=, multiplication *=, and division /=.

There is also an in place assignment operator can assign an array concatenation:

This usage may not be robust and you may be asking for an Error 4127 when you try to append a column to a table. Needs some more experimentation perhaps.

"//" This is a concatenation operator. This makes a rarray (an array of reals): b=3.0//4.1//5.2

The copy operator is used for molecular objects. Normal assignment is fine for normal variables, but molecular objects should, apparently, be created explicitly with the copy command.

The logical operator for testing if something is the same as something else is ==. To test if it’s not the same use !=. Also inequality operators work as expected: <, >, <=, >=.

That all sounds reasonable, but Andrey points out that there can be odd exceptions. For example, in a fresh ICM session the following behavior is observed:

http://www.molsoft.com/man/arithmetics.html All the normal algebraic operators work, +, -, *, /. Note that, like Python, the result of dividing two integers is an integer. This can produce unexpected results:

This happens because 3/4 (both integers) equals 0 which is 0.75 with the decimal (non integer) part discarded.

Arrays do not handle simple arithmetic operators like you might expect. If + concatenated two arrays, what would / do? Instead, when arithmetic operators are used on arrays, a new array is formed by the operation being performed on each value of the array (in order). This allows for things like this:

Note that if you want array concatenation, you can do that with the // operator.

There are also operators for pattern matching, ~ and !~. Here’s an example:

Regular expressions Regular expressions can be used with the functions Match(), Replace(), Index(), Split()

Actually, ICM has several different kinds of regular expressions in different functions and selections. for instance, the Replace function recognizes two different sorts of expressions and its behavior is controlled by the regexp keyword. Sometimes we have to create multiple escape sequences to make selections work, e.g.

if

Conditional statements aren’t too confusing but there are (at least) a couple of modes to consider. First is a single line type of thing like:

The more complete construction is something like this:

You can omit the elseif and/or else clauses as necessary.

An important consideration when using if statements is the nature of the conditional expression. This must evaluate to a "logical" type which can take the value yes or no, both reserved words for signifying this.

The normal way to iterate is to iterate over a range using a for statement.

I don’t know if there are direct ways to iterate over the items in an array, but there are definitely techniques to achieve that effect which are commonly used:

It seems that ICM does not have a do-while type of control structure. The conditional expression of a while statement is always evaluated before any of the body is executed. Under ideal circumstances, that looks like:

To get a do-while effect the commonly used idiom looks like this:

The output looks like:

Note the use of the break key word. The limit flag is used to work in batches of 100 until there are not 100 left and when the remaining are processed.

In this case the while is really being used as a backwards goto. ICM has a goto which can jump forward to a label formatted like LABEL:, but it can’t jump backwards. So the while is used.

FAQ: Is there some kind of iteration using {} expansion somehow?

The most basic way to organize functionality into different containers is to use the call command to import ICM commands contained in other files. This is about like source in Unix shell languages.

Here’s an example:

Note that this hangs for some reason. The quit command may not have obvious behavior in these circumstances.

FAQ: What’s the difference between:

Although ICM has a read command which will load a particular file, there is no shell-style source command per se. Apparently, the entire concept is inadvisable anyway because when ICM is updated, not all external files will necessarily be compatible with the old version.

FAQ: Is there an eval kind of command? Answer: There is. Basically the $ is an eval.

Also consider the following example:

I don’t really know what that does.

Commands are provided by ICM and can not be created or modified by normal users. They take options and arguments (not necessarily in that order). They can not be nested like functions.

FAQ: Is there an exit code equivalent? Answer: Yes. Look at the result of Error() after running a command.

ICM has numerous built in functions. Stylistically, these start with a capital letter and contain parameters (if any) in parentheses. In addition to taking parameters, functions can return values and can be used in assignments and nested expressions.

Functions can also be created by users using the following technique:

This returns:

Here is an example using an arbitrary number of arguments:

Note that function (and macro) parameters must have a special naming scheme. There must be a prefix of s_ for string types, r_ for real types, i_ for integer types, etc. The complete list of how this works is here. However, note that advanced complex types that aren’t listed, for example collections, should use the P_ prefix. The point to all this is that if the parameters are all of different types, then the argument order of the function call can be arbitrary.

FAQ: Is there a way to make a "module" or "package" that can keep name spaces clear? Answer: Use ‘call` to import user defined functions. Functions’ variables are local to them.

Macros tend to have names capitalized like this: sortSeqByLength. Of course there are exceptions (homodel, cool, nice, morph2tz, set_icmff). The show command can help figure out what’s going on with macros.

It can do things like show the contents of a macro. You can also see all the defined macros. For example, show macro shows the whole list of them with their parameters.

It is rumored that, like commands, macros can not be used in direct nested expressions. Functions can like: a( b(x) )

If you need to get rid of some macro that you created, you can do it with:

Also be aware of the set key command which can bind keys to a particular action. For example:

Of course this is already done and if you try to do it again, it will say `key [F1] is already bound.

Also show key to see what keys are already defined.

FAQ: How does one delete or modify key bindings?

You can do show alias to reveal all of the currently defined aliases. You can also do list alias if you like to have options.

alias <name> <action> delete alias <name>

Generally it is not recommended to use aliases in scripts.

FAQ: What’s the difference between an alias and a macro?

You can query the state of environment variables from ICM with the Getenv function. It works like this:

If you’re unsure if an environment variable exists, you can look at something like:

And if you need to set your own environment variables Putenv().

ICM has its own way of doing options that is not exactly like normal Unix scripts. While not great for consistency with other shell scripting arrangements, it is however designed to have less redundant features and is often much less complicated to implement.

I do not think that ICM scripts have a built in way to handle short style options. Handling long options, however, is quite simple.

To use options with ICM scripts you can pass the options to a running ICM session with the -a option when running ICM (the ICM executable does have some short style options, they’re just not available for easy parsing in your scripts).

Here’s the simplest demonstration of argument handling:

Note: You can have arguments like -x=3 if you want, but you have to use "-x" in the Getarg() function. Probably best to just avoid that potential confusion and leave the dashed options for the icm executable.

This makes a more complex example clearer:

The most basic way to read data from a file in ICM is to create a string array with each line of the file being an item in the array. This looks something like:

The keyword comment causes lines that begin with the # character to be skipped.

When reading some kind of data, it is probably best to employ one of ICM’s fancier file reading methods. Here’s a comparison of the two methods. First the file is loaded as text into a string array. You can see that the file has a header row that isn’t part of the data per se.

Next the data is loaded into a table object. This allows its header to be correctly understood and used and the fields correctly parsed.

If you’re reading something astronomically huge you may not wish to have it all load into an ICM object. You may not have enough memory.

FAQ: How does one read data from files? Need to include CSV into tables (read table), text into Sarrays (read sarray), etc. Also need to show how to do something with each line of text as it comes in and not store it in a list. (see limit=n_records)

FAQ: Maybe talk about stream strategy somewhere. Is a stream a type?

The Ask() function can allow a script to pause and ask for user input.

In this case the default value is "5" and it is prompted. The behavior above is from just hitting enter at the prompt. Note that a colon is added so hopefully that is indeed the formatting you want.

FAQ: When using string types, you can use the keyword simple after the default value specification (no additional comma) - what does this do?

Also there is the pause command. Normally you supply an argument specifying how many seconds to pause, however, if you don’t, it will wait until the user hits a key.

Generally the contents of objects can be output by using the print command.

There is also a show command in ICM with 1000s of uses. There is also a list command with some print-like abilities.

ICM provides normal printf mechanisms for outputting explicitly formatted text. There is printf which outputs to stdout and the special variable s_out. There is sprintf which only saves the output in the special variable s_out.

There is fprint which works as follows for outputting directly to files:

FAQ: What will an .icb file be? Can a "session" be saved from a script? Any other odd cases? An entire ICM session can be saved from a script with the following approach:

FAQ: Tables to CSV? See the write command which can do this.

Here’s an example of a table being written to a comma separated value file:

This can be recalled with the following:

If you want the web server to not just deliver the source code of an ICM program, but rather execute it and deliver the results, you must configure that in the web server configuration. There are known places where this is done globally, but when you want to do it in some arbitrary directory, you can use a .htaccess file which allows you to make custom local configuration changes. To have files that end in ".cgi", ".py", or ".icm" execute, create a .htaccess file in the directory where these programs will be served which looks like this:

Some people don’t mind mixing up HTML and proper programming but I find that it makes syntax highlighting terrible and it opens a lot of possibilities for errors trying to match opening and closing tags. Matching tags is the kind of thing a computer should be doing and when writing real programs, you can let the program do this. Here is my function to do this translated to ICM from my Python version.

Running this will produce:

As described in this web page, ICM has the ability to have HTML documents in working memory and to display them in a type of browser (GUI, I presume). This is started with something like:

One nice trick that ICM can do to assist web development is to convert ICM tables into very nice HTML tables. The basic way to do that is the following.

The sort keyword adds some magical stuff to the output so that the table is "sortable". More specifically, it comes with some Javascript code (or references to it) that will resort the table on the client’s browser. This code is based on Stuart Langridge’s "sortable" code. The Molsoft team have a version that lives here.

One problem with this code is that it generates a complete HTML document instead of an HTML fragment. This means that it would, for example, be less convenient to include two sortable tables from ICM tables in a single HTML document.

One way to solve this problem is to parse the output of this show html functionality. This is possible with the following function:

Another way around this limitation is to make your own tables explicitly. This may not be as convenient but sometimes if you want something done right, you have to do it yourself. Here are some functions that can produce simple HTML tables from an ICM table. Note that these assume the HTML tagger function discussed previously.

Pretty much all sane browsers today support the very useful HTML5 canvas tag which allows for composing arbitrary graphics using JavaScript. This has obvious implications for domain specific technical diagrams like molecular diagrams.

The Molsoft function man page has this tantalizing and extremely brief mention of the topic under the String() function

I don’t know exactly what that means. What is an X_chem? Is it different from X_chemarray?

Here is a function which will create the HTML5 canvas tag. Note that if you have multiple ones of these they will all be in the same place. The way I fixed that was to do something like this:

Do this in the loop where Ui is the number of the molecule you’re on. Basically make them all different if you want it to render to different canvases.

Note that you need to specify the size of the canvas too, it’s not optional.

I also found that the executable I used did matter. Some versions of ICM may not have this feature compiled in. Check the error message for String(x y). Look for:

If that is part of the error message for String, then you can be pretty sure it should work.

It is also possible to export chemical as svg.

Note that this requires that you be running ICM in graphical mode (-g).

A discussion of web technologies featuring ICM would not be complete without discussing ActiveICM. This is a browser plugin which can allow full 3-d molecular models to be rendered and manipulated in a browser window. The down side to this is that the plug in can be tricky to get working. Even though it costs no money, the downloading and configuration can limit spontaneity. However, if the investment is made, one is empowered by a truly excellent way to view molecules from a web browser. ActiveICM is vastly better than the popular Java alternatives and as a bonus you don’t need to have Java, the security nightmare, installed.

The team at Molsoft has written some nice applications in JavaScript that shows this off. These might be useful references:

There are many aliases that are predefined so it’s a good idea not to use the same identifier for your stuff. For example, q is an alias for the quit command. To see all the aliases use list alias.

The following key words are built in commands in ICM:

alignSS, aquaSitesBG, calcEnergyStrain, calcEnsembleAver, calcMaps, calcPairSeqIdsFromAli, calcPepHelicity, calcProtUnfoldingEnergy, calcRmsd, calcSeqContent, calc_nosauc, chemSuper3D, convert2Dto3D, convert3Dto3D, convertObject, cool, ds3D, dsCellBox, dsChem, dsCustom, dsPocket, dsPropertySkin, dsPrositePdb, dsRebel, dsSeqPdbOutput, dsSkinLabel, dsStackConf, dsVarLabels, dsXyz, evalSidechainFlex, findFuncMin, findFuncZero, findSymNeighbors, find_related_sequences, gAlignTwoSequences, gCalcArea, gCalcBindingEnergy , gCalcDihedralAngle, homodel, icmCavityFinder, icmMacroShape, icmPmfProfile, icmPocketFinder, loadEDS, loadEDSweb, makeAxisArrow, makeIndexChemDb, makeIndexSwiss, makePdbFromStereo, makePharma, makeSimpleDockObj, makeSimpleModel, mergePdb, mkUniqPdbSequences, modifyGroupSmiles, morph2tz, nice, optimizeHbonds, parrayTo3D, parrayToMol, plot2DSeq, plotBestEnergies, plotCluster, plotFlexibility, plotMatrix, plotRama, plotRose, plotSeqDotMatrix2, plotSeqDotMatrix, plotSeqProperty, predictSeq, prepSwiss, printMatrix, printPostScript, printTorsions, rdBlastOutput, rdSeqTab, refineModel, regul, remarkObj, searchObjSegment, searchPatternDb, searchPatternPdb, searchSeqDb, searchSeqFullPdb, searchSeqPdb, searchSeqProsite, searchSeqSwiss, setResLabel, seticmff, sortSeqByLength, waterDisplacement,

Need an SDF file to work with but you don’t have one? Often if you know what the molecule is you can hunt down it’s SMILES string on Wikipedia and do this.

To take a SMILES string and make a 3D model use the build command.

You can give it a name with name="eto" (mind the quotes on the name). If you don’t name it, it will get a chemical name (like "H2O").

If you need to build a protein or a peptide chain, that is extremely easy to do. Going with the easiest, you can just load a model from PDB.

I’m not sure if this converts from bad PDB representation to proper ICM version. You can also just build the molecule yourself from a peptide sequence. Here’s Lixisenatide, a drug for treating diabetes made from something resembling Gila monster venom.

Selections can be tricky. You can select atoms, residues, basically anything conceivable. Here’s the full documentation on selections. Generally selections look like this.

Here I load a PDB structure and then delete all the atoms that aren’t part of a particular alpha helix I’m interested in.

Double clicking a workspace object will select atoms in an object. And double clicking empty space in the hierarchical workspace list will unselect everything.

Select something with a box using right button and then erase everything but that with this.

Select all? It’s something like this, but probably not exactly.

How to extract the DrugLikeness property from the Predict function. Shown here starting with a SMILES string.

Negative are not drug like, positive are.

Often objects wander around in space and you’d really like them to be pointed the same way in the same place. Here’s what the Expert technique is.

These commands do a good job of putting the stuff on the screen.

Here is a way to get rid of the cartoons ribbon display that usually is popular.