Cloudgizer

Cloudgizer is a free open source tool for building web applications as Apache modules, with emphasis on performance, small-footprint, and more productive and safer programming in C. It combines the ease of scripting with the power of C, improving the resource utilization for cloud applications.

Cloudgizer is licensed under Apache 2 License.

Current stable version is 1.2 (as tagged in git repository).

This site has been last updated on Tue Dec 18 22:10:37 MST 2018.

This is the documentation for the latest Cloudgizer code. If you download older tagged versions from the git repository, each comes with 'docs' directory that contains the documentation matching that version.

Read an article introducing Cloudgizer.

To get started now, read the article on how to install Cloudgizer and build your own application quickly.

About Cloudgizer
    Example source code - learn by example
    License
    Source code & Issue tracking
    Hello world
Basics
    The big picture
    Basic components
    The architecture
    Structure of an application
    Hello World
Download and Installation
    Install Cloudgizer
    Create Example Application installation file
    Install the Example application
    Your own application
        Developing and building application, Cloning applications
        Packaging
        Deploying
    Smoke tests
Getting started
    Configuration, database and debug files
        Configuration file
        Database file
        Debug file
    How to write code
        Markup code block
        Markup usage
        Line continuation and whitespaces
        A typical program
        Errors in your program
    CLD utility
Processing of requests
Constructs
    Comments
    Outputting data
        Whitespaces and empty lines
        Semicolon
        Encoding output
        Ignoring markup
        Outputting preprocessor data to a file
        Outputting HTTP header for a typical web page
        Defining, setting and outputting integers
        Outputting data from a command line program
    Embedded C code
    URL input parameters (GET and POST)
    Cookies, setting and getting
    Database queries
        Basic usage of queries
        Database transactions
        Column Length
        When query has no results
        Force query to produce a single empty row
        Current row in a query result set
        Number of rows and columns in a query result set
        Number of affected rows in DML
        Input parameters to queries
        Column names
        Nesting queries
        Using multiple query texts at run-time (conditional queries)
        Dynamic queries
        Dynamic queries when table structure is unknown
        Result-set data array
        Using query results more than once
        Copying query text in multiple places safely (query shards)
        Re-using query text in multiple places (query fragments)
        Loop control in result set
        Error handling in executing queries
        Getting auto_increment value
        NULL and result column data types
    Executing Cloudgizer program from command line
    Executing external programs
    Sending emails
    Loading URL content (calling a web page) programmatically
    Strings
        Defining and copying strings
        Appending string
        Search and replace string
        Constructing complex strings
    Program flow
        Loops
        Conditional statements (if, else...)
    Files
        File storage
        Permissions and ownership
        Uploading files
        Reading a whole file
        Writing a whole file or appending to a file
        Copying a file
    Web address of server
    Error handling in general
    Include files
    Debugging your code
        Tracing and debugging options
        Finding where program crashed
        Enabling debugging information
        Which shared libraries are loaded?
API
    Memory allocation
        Allocation and garbage collection
        Get size of memory allocated for variable
        Check if allocated memory is valid, overwritten or underwritten
    Configuration parameters
    Tracing and debugging
        Write to trace file
        Which trace file is being written to?
        Using custom tag value for debugging
        Linting your HTML code dynamically
    URL input
        URL input parameters
        Is URL referrer from the same site?
        URL string that is being processed
        URL parameters manipulation
    Cookies
        Data type for storing
        Cookies: get, set, delete, find
    Global data
    Encoding and Decoding, URL encoding, Web encoding, Base64 encoding
    Numbers
        Create string from integer
        Check if string is a positive integer or a number
    Execution of external programs, piping from one program to another
    Output data and HTTP header
        Outputting HTTP header, custom HTTP response headers
        Encoding data for output
        Output text without breaks
        Disable and Enable HTML output, status of enablement
        Send files to the client and output HTTP headers for files
        File not found (404)
        Flush output
    Create new database document and a new associated file
    Strings
        Initialize, Copy, Append strings
        Output data to string
        Change case of strings (upper, lower)
        Substrings
        Trim string
        Break down string into an array of strings based on a delimiter
    Clear Unused variable error
    Error reporting
    Encryption and hashing
        Hashing
        Encrypting data
        Decrypting data
        Generating random string
    Storing and retrieving data in a sequential list (FIFO)
    Executing Cloudgizer program from command line (Batch processing)
        Using batch processing
        Exit code for batch processing
    POST URL (web call)
    Files
        File storage
        Get file size
        Copy files
        Read entire file into a string variable
        Write whole file from a string
        Append to file from a string
        Lock a file
        Get application home directory
        Check if directory
        Temporary files
    Send email
    Queries and Transactions
        SELECT from the database
        Get value from auto-increment column
        Execute any SQL other than SELECT, including DDL statements
        Check for open transaction
    Get miscellaneous
        Get web address for forms and links
        Get Base URL of server
        Get version
        Get current, past or future time, including GMT
        Get current time
        Get application name
        Get environment variable (web or command shell)

About Cloudgizer

Cloudgizer is a free open source tool for building web applications as Apache modules in C programming language, licensed under Apache 2 License..

Example source code - learn by example


Browse the source code for both Example application and Rentomy to learn by example, get a feel for developing with Cloudgizer and to understand better the usage patterns of markups and API.

License


Cloudgizer is licensed under Apache License, Version 2.0.

Source code & Issue tracking


Source code is available at Cloudgizer git repository.

Older Cloudgizer versions stable enough for download and installation are tagged in git.

Report issues or questions - go to Issue tracker.

Hello world


Here is a program that outputs 'Hello World!' to the browser (and here' a database-enabled one):
#include "cld.h"

void home()
{
   /*<
    output-http-header

    Hello World!
   >*/
}

Basics

The big picture


Cloudgizer is a tool for building web applications as Apache modules in C language enhanced with simple markup, with emphasis on performance, small-footprint, and more productive and safer programming in C.

The programmer writes simple markup language mixed with C code, which is then translated entirely into C code and compiled natively as Apache module. The resulting application is fast and takes less memory, as there are no interpreters or virtual machines.

Features include easy markups to use MariaDB database, HTML input parameters, cookies, simpler outputting of web pages, files storage and manipulation, encryption, encoding, program execution, web calls, safer and easier string manipulation etc. - the list is too long to place in one sentence. Overall Cloudgizer does a lot of stuff for you that you'd otherwise need to do yourself.

A memory garbage collection system and memory overwrite/underwrite detection comes in handy for program stability. The same goes for string and memory handling markups to help write applications that won't crash.

Also included is an application packaging system and an automated application installer. This makes rollout of products and release cycle more manageable.

Cloudgizer source files have extension .v.

Cloudgizer pre-compiler (cld program) will turn your .v files into .c files, ready for compilation as pure C programs.

Then, your program will be compiled and linked with Apache web server on RH/Centos systems. It links with Apache as an Apache module in a "prefork" configuration. It does the work of communicating with Apache, and it makes it easier to write high-performance/small-footprint web programs in C.

Cloudgizer is not designed to be thread-safe as it works in a "prefork" configuration of Apache.

You can also build command-line programs. The same program can serve as both command-line utility and a web program linked with Apache.

Cloudgizer works with RedHat/Centos 7 operating system, Apache web server and mariaDB database.

Basic components


These are the files installed:


All scripts, the makefile and the command line utility are located in /usr/bin directory. Include file is in /usr/local/include/cld directory. Libraries are in /usr/local/lib/cld directory.

The architecture


Apache server is the container which holds your Cloudgizer applications as Apache modules. The module mechanism allows for fast response times and very little overhead.

Apache server can house many Cloudgizer applications, each being a separate Apache module. These separate modules are generally entirely different applications.

Apache handles the infrastructure part while your Cloudgizer code makes up the applications themselves.

All Cloudgizer applications are hosted under a single Linux user, with each application within its own subdirectory under the user's home directory. The Apache server user is set to this Linux user during the application installation - it is the currently logged-on user during installation. Make sure to install all applications under the same Linux user. Set the Apache's DocumentRoot to be accessible by this user.

Each Cloudgizer application has its own database schema within a MariaDB database server.

Structure of an application


Each application is installed in its own directory, known as the application home directory. If application name (given by CLD_APP_NAME in appinfo file during installation) is 'your_app_name', then application home directory is /home/user/your_app_name.

The following are important files your project must have - they all have default implementation in the Example Application:


In short, you must have the following implemented for an application to work:


The rest is you implementing the handler functions for different kinds of requests. To get started without writing everything from scratch, you should take the Example application and change it into your own.

Hello World


This is the Hello World example that is included in the Example application. It reads data from the database and displays it in the browser. It exemplifies the coding style of Cloudgizer, which is C code with markup. Markup code is enclosed in /*< and >*/ tags:
#include "cld.h"

void home()
{
   /*<

    output-http-header

    Hello World!

    run-query#get_db="select hi from hello"
       query-result#get_db, hi as define db_result
       print-web db_result
       <br/>
    end-query
    <hr/>

    >*/
}

Download and Installation

You need RedHat/Centos 7 on a dedicated server or a VPS (virtual private server) and Internet connectivity.

You'll first get the source code from git repository, then install from source. The benefit of source distribution like this is that you will get the highest possible performance by compiling on your specific hardware.

Install Cloudgizer

  • Setup tools, libraries and the database (if you are only upgrading, for example due to a bug fix in Cloudgizer, then you may run setup_cld only, see below):
    sudo ./setup_env
    sudo ./setup_maria
    sudo ./setup_postfix
    Note: Setup process will create MariaDB database and you will be asked to create root database password and secure the database- this is a MariaDB setup script and it may change. The setup will also install other needed software and also setup Postfix for sending emails using TLS encryption - you will be asked to provide the domain name (such as mydomain.com) and the server name (such as myserver.mydomain.com).

  • Setup Cloudgizer:
    ./setup_cld

  • To verify installation, run cld tool (you should see help) and check that documentation is there:
    cld

    ls docs/index.html
    To get started on writing your own application using Cloudgizer, first install the Example application and build on it. Once you create your own application, you would package it and then install it on customer's site. For these steps, see further in this documentation.

    Create Example Application installation file

    Follow these instructions to make the installation file from source code. You must have Cloudgizer installed first.


  • Create installation file:
    cldpackapp

  • Now you have example.tar.gz file that you can use to install Example Application anywhere. See install the Example application.

    Install the Example application

    Once you have created Example application installation file (i.e. example.tar.gz file), get started by installing the Example application. This is required to build an application of your own.

    The following descibes installing from example.tar.gz file.
    #Choose a user for Cloudgizer applications (create one if needed). We will use 'cld' as user name. Login as that user. Apache user will be changed to this user during installation.
    su - cld

    # Unpack Example application
    sudo rm -rf deploy
    tar xvfzm example.tar.gz
    cd deploy

    #Open file appinfo. This file describes the application. Set these variables:

    #After setting these variables, run:
    cldgoapp create
    To test the installation, note the URLs displayed at the end of installation. They include examples and smoke tests.

    In general, for any URL your application may use, the URL's path is always go.your_app_name, where your_app_name is the application name, in this case 'example'. Application name is always the value of CLD_APP_NAME in appinfo file.

    After you have successfully installed the Example application, use it as a base code for your own. You'd probably remove all request handlers except Hello World and start from there.

    Your own application

    Developing and building application, Cloning applications


    If you are a developer, you'd install the Example application first and then build your application by modifying it to suit your needs.

    Once you install the Example application you would clone it into a new application, under the name of your choosing, and develop your application from there.

    You can clone any existing application (including the Example) by simply changing its name and then creating an installation file.

    To clone an existing application, you'd change CLD_APP_NAME in appinfo file and run cldpackapp to create an installation file for your new application. For example, if you change CLD_APP_NAME to your_app_name and then run cldpackapp, you'd create your_app_name.tar.gz installation file.

    See Packaging and Deploying for more information on cldpackapp and on deploying the installation file it produces.

    To build an already installed application, use cldbuild followed by restarting the web server:
    cldbuild
    sudo service httpd restart
    Use this if you're developing Cloudgizer applications, or in testing.

    You would use the above cldbuild followed by web server restart whenever you change your source code files, in order to test the changes. If you want to rebuild all source files, for example if you change database tables, use cldbuild clean to touch all source code files and then cldbuild to rebuild them.

    Most of the time you'd change your source files only (such as .v files). From time to time you'd make more structural changes as your application grows. For example, when changing the database tables, adding or removing files (such as CSS, images, cron jobs) etc., make sure you test your installation. This means your create.sh (and update.sh for upgrades) must be reflecting all those changes in the deployed application. Use cldpackapp to package your application, copy it to a test environment and test your application from scratch using cldgoapp.

    Packaging

    Once your application is ready to be installed at customer site, package the application. The appinfo file is the configuration file for the installation process, and it should contain any variables your installation needs. Do not write appinfo from scratch - always start from the file provided in the Example application and build on it. Script cldpackapp will package your application by producing your_app_name.tar.gz file your customer can install by using cldgoapp.

    Most of the stuff done during installation is the same for any application. The application-specific part of installation is done through create.sh file. This is a script you provide in which you can setup anything that installation process can't do, for example you can create database tables or any other custom setup your application needs. If the application is being updated (as opposed to created for the very first time), you would create update.sh file instead and write your application-specific code there. You can use CLD_APP_INSTALL_DIR in these shell scripts to locate files in the installation.

    appinfo file is a configuration file for the installation. It always has these variables: CLD_SERVER, CLD_APP_NAME, CLD_EMAIL, CLD_DB_ROOT_PWD and CLD_DB_APP_PWD - these variables are set by the customer during installation. Your responsibility is to set the following variables in appinfo:

    Once you have appinfo file as described above, as well as create.sh/update.sh files, create a deployment file:
    cldpackapp
    This will create your_app_name.tar.gz installation file.

    Check out the source code of the Example application.

    Deploying

    At customer's site unpack the installation file:
    sudo rm -rf deploy
    tar xzvfm your_app_name.tar.gz
    cd deploy

    # Edit appinfo file, set these variable to match customer's environment:

    #
    # Use one of the following to install:
    #

    # to create application
    cldgoapp create

    # to update application
    cldgoapp update
    Application is always deployed from binaries. If you distribute source code, then the end-user can recompile the source code by using cldbuild script.

    Any variable that has PWD in the name will be cleared upon installation for security reasons.

    Smoke tests

    A battery of tests designed to shake out any obvious issues is incorporated in the Example application. Install it and run as prescribed - all tests run and display results in web browser. Typically each test will show OKAY if succeeded, or otherwise describe what is the desired outcome.

    Getting started

    Configuration, database and debug files

    These files direct Cloudgizer in some important aspects. Note that these are setup by cldgoapp script based on your appinfo file. cldgoapp script installs a Cloudgizer application. Your own create.sh script (a customizable part of installation) can further change these files.

    Configuration file

    A configuration file (named config) in the application home directory typically has something similar to this
       version=7
       web_address=http://192.168.0.11
       email_address="Hellocld" <root@localhost>
       application_name=Hellocld
       max_upload_size=10000000
       mariadb_socket=/var/lib/mysql/mysql.sock
       ignore_mismatch=no
    version determines the application version. Typically it is used in constructed URL to force refreshment of cached files, but it can be used for any other versioning purpose.
    web_address contains the server address where the application runs on, and is a base URL for Cloudgizer requests. You can use http:// or https://.
    email_address is the email where status emails would be sent (for example in case of a program crash) and it can be used for any other emailing purpose by the application.
    application_name is the name of application - it can be any name that is 16 bytes or smaller, composed of alphanumeric characters and  underscore, cannot start with a digit and cannot be 'deploy'.
    max_upload_size is the maximum size of an upload file - uploading larger file will invoke predefined  file_too_large function, implemented by you.
    mariadb_socket is the database identification, a means to connect to the database.
    ignore_mismatch is by default "no", meaning that if shared library used to build application doesn't match what's installed on deployment server, stop the program. If "yes", skip this check and proceed. Use "yes" with caution and only if you know why you're doing it.

    You can also define user parameters, which are always precedeed by _ (an underscore).

    You can use all of these parameters in your code, see configuration parameters API.
    If you need to use a Cloudgizer application home directory (for example in a user parameter), use tilde (~) sign, for example:
    _my_user_directory = ~/user_dir

    Database file

    .db file is located in the application's home directory and it contains database login information. By default its permissions are set to 600 - do not change this. It has these four lines:
       localhost
       your_app_name (i.e. the CLD_APP_NAME from appinfo, the database user name)
       app_db_password ( i.e. the password you chose,  CLD_DB_APP_PWD)
       your_app_name (i.e. the name of the database created, which is the same as the database user name, which is CLD_APP_NAME in appinfo)

    Debug file

    A debug file (named debug in trace directory) determines the application behavior with respect to debugging. Take a look at debug file options for more details.

    How to write code

    Markup code block

    Markup code block is written between /*< and >*/. It is a comment technically, but cld utility will parse it and generate C code for it. For example:
    void my_func ()
    {
       char *my_var = "hello";

       //
       // The following comment-block is Cloudgizer code.
       // It is processed by cld command line tool to create C code.
       // In this case very simple text outputting.
       //
       /*<
       Any free text that's not a markup is output.
       <hr/>
       print-web my_var
       <hr/>
       >*/
    }
    In this example, the output of the program (meaning output to web browser) is:

    Any free text that's not a markup is output.
    hello


    The actual C code generated by Cloudgizer for this will look like:
    void my_func ()
    {
       char *my_var = "hello";

       cld_printf (CLD_NOENC, "Any free text that's not a markup is output.");
       cld_printf (CLD_NOENC, "<hr/>\n");
       cld_printf (CLD_NOENC, "%s", my_var);
       cld_printf (CLD_NOENC, "<hr/>\n");
    }
    So you can write the web output by simply writing the HTML code (or any other code, such as Javascript, or anything else that works for you). In this documentation, we'll talk about "HTML code" for simplicity, but when we say "HTML code" it means any kind of web code.

    Anything else that is not HTML code, you will use markups such as print-web to output variables, run-query# to run database queries, input-param to get URL GET/POST input parameters etc.

    If /*< and >*/ are not feasible, you can use /*CLD_BEGIN and CLD_END*/ as markup boundaries.

    The markup beginning (i.e. /*<) must be the beginning of the line and markup ending (i.e. >*/) must be the ending of the line.

    Markup usage


    The markup (such as print-web) can be used in-between <? and ?> or without them in most cases, but not all. For example:
    print-web my_var
    is just fine on a single line. But if you want to combine it with other output on the same line, as in:
    This is variable <?print-web my_var?>
    then you must write the markup within <? and ?>. You can also always use these brackets if you prefer - if you are starting with Cloudgizer, it may help you catch errors sooner. Consider if you wrote:
    printa-web my_var
    In this case, printa-web is not a valid markup, even though you intended it to be. The output from this will be:

    printa-web my_var

    i.e. the literal output of what you wrote.  But if you wrote:
    <?printa-web my_var?>
    you would get an error. One might say that seeing your code in the output is a good enough way to find the error in fix it, but using markup will detect an error earlier - however the code might look a little less verbose.  

    Line continuation and whitespaces


    To continue a line of code over several lines, use a backslash (\). It can be used anywhere on the line, including in the middle of a string. Example:
    run-query#my_query = "select name \
                               from employee \
                               where name like '%'"
    is the same as:
    run-query#my_query = "select name from employee where name like '%'"
    Use only spaces as separators in markup code, i.e. do not use tabs and other whitespaces. A line may begin and end with tabs but do not use tabs within markup code itself.

    A typical program


    Example application provided with installation provides a good starting point. You can take it and grow it into your application, it is meant to be used that way.

    Generally, you will get a request via cld_handle_request function. You implement this function (example is provided), and based on some input parameter (typically a page name specified in a URL), you'd call other functions. Most of these other functions would be implemented in their own separate files, thus typically each page would have its own implementation file.

    These other functions are conceptually just like Hello World example. They usually query database and do some work for you. They may write files, read files, process data, generate HTML output, show images and documents and anything else you need.

    You will notice that you generally shouldn't free the memory you allocate - it will be safely freed at the end of a request. Most of the time you won't need to allocate any memory at all, and that's a good thing. The markups chosen are what's usually needed. If something isn't there, you can use the API provided. If that's not enough, write your own C code for it - you can do just about anything in C.

    If you allocate memory on your own always use Cloudgizer memory allocation API such as cld_malloc for example, since all memory allocated this way is automatically released at the end of the request.

    Errors in your program


    Cloudgizer tries to find as many errors in your program as possible. This also means uncovering some errors that might otherwise find their way into run-time. For example, all your non-dynamic SQL (which hopefully is all the SQL you'd ever use) is checked at run-time with the database, so for example syntax errors and using columns that do not exist are likely to be caught at compile time. Cloudgizer will try to figure out if you forgot to use c markup for the C code within the markup, which can cause code not to execute (and instead display) if you've forgotten it.

    The code compiled by C compiler is generated from your original .v file. The error reporting from the C compiler (gcc) will show line numbers from your original code, not the generated code, sparing you the hunt for the actual line in .v file that caused an error.

    At run-time, you'll have a number of facilities available for debugging, which is explained here in this documentation. You can learn how to use gdb to debug your program, how to check the validity of your generated XHTML at run-time, how to use tracing etc.

    CLD utility


    cld utility translates your .v file into .c file which is then compiled and linked with your program. If you type cld at the command line, you will get this help:

       Name
       
       Cloudgizer code generator, markup language and application server API, version [1.2]
       
       Description
       
       Cloudgizer is a tool for building Web applications in C that run as modules on Apache web server on RedHat/Centos. It supports mariaDB database by using LGPL mariaDB client that enables connectivity to mariaDB database. Each application runs under the same Apache web server user, under its own directory (i.e. application's home directory).
       
       Synopsis
       
       cld [<input-file-name.v>] [<command-line-options>]
       
       Options
       
       -help
           Display this help.
       
       -out <output-file-name.c>
           Write generated code to output file <output-file-name.c>. If this option is not used, generated code is written to stdout (standard output).
       
       -main
           Generate main C code. This option cannot be used when <input-file-name.c> is specified, i.e. either C code is generated for input-file-name.c or the main() function C code is generated.
       
       -cmd
           Generate C code for use as a standalone program (a command line program), rather than as an Apache module (Apache Mod) program which is the default. This option can only be used together with -main.
       
       -mariasock <socket-file-location>
           Specify the location of the mariaDB socket file, used by the database server (socket option in my.cnf).
       
       -v
           Print out verbose information about what is being done.
       
       -urlencode <string>
           Prints URL encoded <string>.
       
       -webencode <string>
           Prints web encoded <string>.
       

       COPYRIGHT AND LICENSE
       
       Copyright (c) 2017 Dasoftver LLC (on the web at https://bitbucket.org/dasoftver/cloudgizer).
       Cloudgizer is free Open Source Software licensed under Apache License 2. Cloudgizer is "AS IS" without warranties or guarantees of any kind.

    Processing of requests


    The cld_handle_request function (the "request processor") is called from generated code to handle incoming requests and it must be implemented. The Example application already has an implementation.

    A simple implementation may look like:
    void cld_handle_request()
    {
       /*<
       input-param page
       
       if-string page="home"
           c home ();
       else-if-string page="other"
           c other ();
       else
           report-error "Unrecognized page %s", page
       end-if
       >*/
    }

    The request processor function is called both for web requests (through Apache module mechanism), or for command line program execution. You can use any input parameters to differentiate input requests ('page' is used here as an example only).

    Typically file cld_handle_request.v is created to hold the request processor code.

    Constructs


    The markups are always within /*<  ... >*/ - that's a markup block. They are parsed by cld utility and C code is produced where comment was before. You will see that you can also have pure C code within /*< ... >*/, so while /*< ... >*/ is within a C code already, you can have C code again within it. It's useful because it's easier to write code this way - sometimes you'll need to have small snippets of C code (such as function calls or small blocks of code) to augment your markups.

    Comments


    There are a few ways to comment your code while in a markup block.

    Use either:

    Outputting data

    Whitespaces and empty lines


    Semicolon


    If semicolon is the last outputted character on the line, it must be escaped:
    Hello world<?;?>
    This is because any line that ends with semicolon is expected to be C code.

    If you have lots of semicolons (for example in Javascript code), you can use start-verbatim/end-verbatim markups, in which case nothing in the output (including semicolon) is interpreted.

    Encoding output


    You can output data without any encoding, by web encoding or URL encoding:

    Ignoring markup


    Common outputting of HTML code is just to write it without any markup, as in this code:
    Hello<hr/>
    This is HTML<br/>
    will produce:

    Hello
    This is HTML

    If your output happens to coincide with a markup, such as for example, if it begins with if, then you could either output with a print-web (for example):
    print-web "if here won't be treated as if markup"
    or use w markup, which does nothing, but it positions whatever comes next as not being the first on the line, which precludes it from being interpreted as markup, for example:
    w if here won't be treated as if markup
    resulting in output:

    if here won't be treated as if markup

    If you have a large segment of text such as JavaScript that you just want to output to web page without worrying about any Cloudgizer markups being interpreted, or about trailing semicolons, use start-verbatim and end-verbatim:
    start-verbatim
    if here won't be treated as if markup<br/>
    for here won't be treated as for markup<br/>
    end-verbatim
    resulting in output:

    if here won't be treated as if markup
    for here won't be treated as for markup

    When start-verbatim is used, anything up to end-verbatim is output as if with print-noenc.

    Outputting preprocessor data to a file

    Use preprocessor-output# to output data during processing of source code files. For example:
    preprocessor-output#indexes employee:name
    run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee where name='John'"
    end-query
    This will write to 'indexes.clo' file the line after the file name, in this case 'employee:name'. Such output can be used to determine what indexes are needed on each table used. In general any information can be added to any file. The purpose is to help with static analysis of code as in aforementioned example.

    Outputting HTTP header for a typical web page


    Before outputting any text to a web page, HTTP headers need to be sent out:
    output-http-header
    print-web "Hello world"
    <hr/>
    The header will contain any cookies that are present, meaning that had been received from the client, added and have not been deleted. Because this is the dynamic output from the program, the client is instructed not to cache.

    If you want to send custom headers (for instance change content type, caching, status or any other options), you can use custom header API.

    Defining, setting and outputting integers

    Integers can be used via following markups:
    // define integer with default value of 0
    define-int new_int

    set-int new_int=10

    // define and set the value of integer with any expression
    define-int my_int=4*new_int

    set-int my_int=my_int+10
    To output an integer:
    print-int my_int
    To output a long:
    print-long my_long

    Outputting data from a command line program

    To output to standard output, use:
    print-out "Today is a good %s", "day"
    To output to standard error, use:
    print-error "Today is a good %s", "day"
    These constructs can be used only from a command line program. The syntax of the format and the data output is the same as for C printf() function.

    Embedded C code

    Use either:



    With c, only one line of C code can be written that accompanies c.
    With start-c...end-c, multiple lines of C code can be written. This C code is NOT placed in curly braces.

    URL input parameters (GET and POST)

    Input parameters from an incoming request are obtained by using input-param:
    input-param par1
    input-param par2

    Input parameter par1 has value
    print-web par1
    , and input parameter par2 has value
    print-web par2
    This will create two string variables par1 and par2 and store input parameters par1 and par2 into them, respectively. For example, if URL that led here was:
    https://mywebsite.com/go.your_app_name?par1=value1&par2=value2
    then string variable par1 will have value "value1" and par2 will have value "value2", and the output from the above code is:

    Input parameter par1 has value value1, and input parameter par2 has value value2

    input-param works the same for both GET and POST requests. Input parameters are trimmed for whitespace (both on left and right).

    Input parameter name can be made up of alphanumeric characters or underscore only and cannot start with a digit.

    Cookies, setting and getting


    Use cld_time API function to produce time suitable for cookies.

    Database queries


    You can SELECT, INSERT, UPDATE or DELETE database table, retrieve results, check for errors. You can not perform any other SQL statements with markups, for example DDL statements - if you want that, use the API.

    You can connect to a single database, specified in .db file.

    Writing queries, especially when many queries have common text, is supported with additional features such as query fragments and query shards.

    Basic usage of queries



    query-result# can be used with noencode (no encoding), urlencode (URL encoding) or webencode (web encoding), depending on where it used. For example, if it is used in construction of a <a href ..> link in the list of parameters, urlencode should be used. Default is webencode. These can be combined with define to store the value into it.
    query-result#my_query,name define my_var urlencode
    This will URL encode value of column 'name', then create string variable my_var and store the value into the string variable.

    Database transactions


    Once you start transaction with begin-transaction, you must either commit it with commit-transaction or rollback with rollback-transaction. If you do neither, your transaction will be rolled back once control is passed out of your request handler. Opening a transaction and leaving without committing or a rollback is a bug in your program.  If you want to catch this kind of situation and possibly either commit or rollback, use cld_check_transaction() API function (possibly within a function dispatcher in cld_handle_request(), after the request handling), where you can detect this and either rollback, commit, or error out.

    Column Length


    To output column length for a column in a table:
    <input name="my_column" type="text" value="" size="30" maxlength="<?column-length employee.name?>">

    // To get column column length into a string variable:
    column-length employee.name as define my_column_length
    // Or define a string variable first:
    define-string my_column_length
    column-length employee.name as my_column_length

    <input name="my_column" type="text" value="" size="30" maxlength="<?print-web my_column_length?>">
    The length obtained is the maximum number of characters in a column when it is output as a string. The supported database types for use in this markup are varchar, char and number types.

    When query has no results

    If your query returns no rows and you want still to get a single row made of empty strings for each column, use use-no-result:
    define-query#my_query
    use-no-result#my_query
    run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee where 1=2"
       query-result#my_query, name
       ,
       query-result#my_query, yearOfHire
    end-query
    This will produce output:

    ,

    meaning empty name, then a comma, and then an empty year of hire.

    Force query to produce a single empty row

    Sometimes you may want to get a single row made of empty strings for each column, in which case use create-empty-row:
    define-query#my_query
    create-empty-row#my_query
    run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
       query-result#my_query, name
       ,
       query-result#my_query, yearOfHire
    end-query
    This will produce output:

    ,

    meaning empty name, then a comma, and then an empty year of hire. Even though query would have produced some rows, you will still get a single empty row. The query is not actually executed to do this.

    You may want to use this to conditionally either execute the query, or just produce an empty row, for example:
    ...
    define-query#my_query
    if execute_query!=1
       create-empty-row#my_query
    end-if
    run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
       query-result#my_query, name
       ,
       query-result#my_query, yearOfHire
    end-query
    In this case, the same code is used to either show the data from the database or an empty row, such as if you wanted to present a blank form to be filled out.

    Current row in a query result set

    Use current-row# to get the current row number, starting with 1.
    run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
       Row #<?current-row#my_query?><br/>
       query-result#my_query, name
       ,
       query-result#my_query, yearOfHire
       <br/>
    end-query
    This produces output:
    Row #1
    Linda,2001
    Row #2
    John,2003

    You can also store current row in a string variable first:
    run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
       // You can define variable on the spot
       current-row#my_query as define curr_row
       // Or you can define it first:
       define-string curr_row
       current-row#my_query as curr_row

       Row #<?print-web curr_row?><br/>
       query-result#my_query, name
       ,
       query-result#my_query, yearOfHire
       <br/>
    end-query
    The output produced is the same.

    Number of rows and columns in a query result set

    Use row-count# and column-count# to get the number of rows and columns returned by the query.
    run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
       Data is <?query-result#my_query, name?>, <?query-result#my_query, yearOfHire?> (row <?current-row#my_query?> out of <?row-count#my_query?> with <?column-count#my_query?> columns) <br/>
    end-query
    Number of rows: <?row-count#my_query?><br/>
    Number of columns: <?column-count#my_query?><br/>
    This produces output:
    Data is Linda, 2001 (row 1 out of 2 with 2 columns)
    Data is John, 2003 (row 2 out of 2 with 2 columns)
    Number of rows: 2
    Number of columns: 2

    You can also store number of rows and columns in a string variable:
    run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
    end-query
    // Define variable on the spot
    row-count#my_query as define row_count
    column-count#my_query as define col_count
    // Or define a string variable first
    define-string row_count
    define-string col_count
    row-count#my_query as row_count
    column-count#my_query as col_count

    if atol(row_count)!=2
       <div>Number of rows is not 2, but rather it is <?print-web row_count?>!</div>
    end-if
    If number of rows in the table is 3, the following output is produced:

    Number of rows is not 2, but rather it is 3!

    Note that row-count and column-count can be used within run-query loop and also right outside of it as well.

    Number of affected rows in DML

    Use query-result#query_name,affected_rows to get the number of rows affected in INSERT, UPDATE or DELETE.
    run-query#my_query="insert into employee (name, dateOfHire) values ('Terry', now())"
       Number of rows inserted is <?query-result#my_query,affected_rows?>
       // You can also get number of rows into a string variable
       query-result#my_query,affected_rows as define nrows
       if-string nrows!="1"
           report-error "Cannot insert data!"
       end-if
    end-query
    In the above example, we check the number of rows affected (in this case inserted). Consult database manual for more information about number of rows affected - it may not always be what you'd expect.

    Input parameters to queries


    Query input parameters are specified via <?...?> markup. Do not confuse query input parameters (data used in SQL statements) with web input parameters (which are name/value pairs from input URL specified with input-param).
    input-param employee_name
    input-param year_of_hire

    run-query#my_query="insert into employee (name, yearOfHire) values (<?employee_name?>, <?year_of_hire?>)"
       New employee ID is <?query-result#my_query,insert_id?>
    end-query
    In the above example, web input parameters employee_name and year_of_hire (specified in the URL such as yourserver/go.your_application?employee_name=...&year_of_hire=...) are used as query input parameters in INSERT statements.

    Query input parameters are always strings (i.e. char*) by type and can be any C expression, for example:
    run-query#my_query="insert into employee (name, yearOfHire) values (<?by_employee==1 ? entity_name : employee_name?>, <?calculateYearOfHire() + 1?>)"
    In the above example, both employee name and year of hire are results of C expressions.

    Another example:
    run-query#my_query="select yearOfHire from employee where name=<?employee_name?>"
       query-result#my_query,yearOfHire as define year_of_hire
       Employee found, hired in year: <?print-web year_of_hire?>
    end-query
    Query input parameters are sanitized against SQL injection attacks, and they are also trimmed on both left and right if trim-query-input is in effect.

    To trim all query input parameters, use:
    trim-query-input
    To not trim, use:
    no-trim-query-input
    Any query occurring at run-time after either markup will be under its effect. By default, all query input parameters are not trimmed.

    Column names

    You can obtain the names of query columns by using column-names# markup:
    run-query#some_query="select * from employee"
       column-count#some_query as define col_count
       column-names#some_query as define col_names
       define-int i
       for i = 0; i < atoi(col_count); i++
           Column #<?print-web i?> has name <?print-web col_names[i]?>)
       end-for
    end-query
    Note that column-names# means the names of query columns, which may or may not match any actual table column names, since query outputs can have aliases (and they should have them if the output is computed). In the following example, the output will be 'employeeFirstName' and 'employeeLastName' as they are aliases:
    run-query#some_query="select firstName employeeFirstName, lastName employeeLastName from employee"
       column-names#some_query as define col_names
        Column names are <?print-web col_names[0]?>  and <?print-web col_names[1]?>
       end-for
    end-query
    column-names# must have as clause. In the above example, the variable 'col_names' is defined automatically, but you can also define it yourself and omit the define clause:
    c char **col_names;
    column-names#some_query as col_names
    Note that column-names can be used within a query loop (like we did above, such as within run-query/end-query for example), or just outside of it.

    Nesting queries


    Queries can be nested, for example:
    run-query#query1="select id from user"
       query-result#query1,id as define id
       run-query#query2="select timeout from settings where id=<?id?>"
           query-result#query2,timeout
       end-query
    end-query
    In this example, query2 is nested within query1, using its results. Note that query-result# for a query can be used only directly within a query loop, and not within another query's loop. For instance, query-result#query1,id cannot be used within a query loop for query2. This restriction is solely in place to prevent errors that are hard to track, such as having same column names under different queries. It is always possible to use query results directly within the loop, and is a better programming practice.

    Using multiple query texts at run-time (conditional queries)

    A query can have different text depending on run-time conditions. The text of the query is still static though:
    input-param my_name_filter
    input-param my_action


    define-query#my_query

    // depending on a run-time conditional, start two different queries
    if-string my_action="show_all_records"
       start-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
    else
       start-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee where name like concat(<?my_name_filter?>,'%')"
    end-if

    // loop through a query, regardless of which one we will execute in loop-query
    loop-query#my_query
       Name is <?query-result#my_query, name?>, year of hire is <?query-result#my_query, yearOfHire?><br/>
    end-query
    If the input parameter for my_action is 'show_all_records' (for example URL is https://mysite.com/go.your_app_name?my_action=show_all_records), the output is:

    Name is Linda, year of hire is 2001
    Name is John, year of hire is 2003

    If the input parameter for my_action is empty and input parameter for my_name_filter is 'L' (for example URL is https://mysite.com/go.your_app_name?my_name_filter=L), the output is:

    Name is Linda, year of hire is 2001

    For conditional queries with the same name, even though the query texts (the SQL texts) can be different and they can have different input parameters, they must have the same output, because the results sets of all such query texts are used in a single loop-query loop. For example, in the above conditional query my_query, one query text has no input parameters, and the other one has a single input parameter. However, both  texts have the exact same output ('name' and 'yearOfHire').

    Dynamic queries

    A dynamic query has text that is not known at compile time, i.e. it is produced at run-time. Dynamic query cannot be checked at compile time. Dynamic query should be used rarely and only when absolutely necessary. The following is an example and would be normally written as a conditional query.
    // Get input parameters
    input-param employee_name
    input-param employee_year

    // The base query that's present either way
    define-string query_text="select name, extract(year from dateOfHire) from employee where dateOfHire>'%s'-01-01 0:0:0' "

    // State that dynamic query comes from string variable query_text and specify what are output column names
    define-dynamic-query#my_query=query_text with-output name, yearOfHire

    // Query text that's added to the base query if employee_name is specified
    define-string where_clause=" and name='%s'"

    // Construct the query text at run-time. Add appropriate parameters.
    if-string employee_name!=""
       append-string where_clause to query_text
       // We have two input parameters in this case
       add-query-input#my_query: employee_year, employee_name
    else
       // We have one input parameter in this case
       add-query-input#my_query: employee_year
    end-if

    // Execute query. Only at this point do we actually use the dynamic text of the query.
    run-query#my_query
       Employee <?query-result#my_query, name?> was hired in year <?query-result#my_query,yearOfHire?><br/>
    end-query
    The text of query is a variable query_text. This variable is constructed at run-time based on whether input parameter employee_name is provided or not. Clause with-output is used to specify what are the names of output columns - their order and names must match the query's output columns.

    Since the query is dynamic, the query's input variables are not known and must be specified. Input variable is specified with '%s'. The actual input variables are provided in add-query-input# clause.

    If the input parameter for employee_name is Linda (for example URL is https://mysite.com/go.your_app_name?employee_year=1990&employee_name=Linda), the output is:

    Employee Linda was hired in year 2001

    If the input parameter for employee_name is empty (for example URL is https://mysite.com/go.your_app_name?employee_year=1990&employee_name=   or just    https://mysite.com/go.your_app_name?employee_year=1990), the output is:

    Employee Linda was hired in year 2001
    Employee John was hired in year 2003

    Dynamic queries when table structure is unknown


    In some cases, you would not know the output columns of a dynamic query. For example, a dynamic query could be constructed in form of 'SELECT * from ...'. In general it may be beneficial in certain situations to obtain query results even if you don't know (or is cumbersome to obtain) the query output columns. In these cases you can use with-unknown-output markup, as in this example:
    // Get all tables from current database
    run-query#all_tabs="select table_name from information_schema.tables where table_schema=database()"
       query-result#all_tabs, table_name as define table_name

       // Construct the run-time text of a dynamic query
       write-string define qry_txt
       select * from <?print-noenc table_name?>
       end-write-string

       // Define dynamic query to be the text above and specify that we don't know column names
       define-dynamic-query#get_tab = qry_txt with-unknown-output

       // Run query
       run-query#get_tab
       end-query

       // Get some table metadata and also the actual table data
       row-count#get_tab as define row_count
       column-count#get_tab as define col_count
       column-names#get_tab as define col_names
       column-data#get_tab as define col_data

       // Display data
       define-int i
       define-int j
       for j = 0; j <atol(row_count); j++
           // Display columns for each row
           for i = 0; i <atol(col_count); i++
               // Show column names and column data (which is a flat array that spans all column and all rows)
               print-out "colname %s, coldata %s\n", col_names[i], col_data[j*atol(col_count)+i]
           end-for
       end-for
    end-query
    In this example, we don't know table names (or their columns) because we get the list of tables from the current database. Query text is constructed on the fly and we define dynamic query with-unknown-output because we don't know the output column names. After running the query we obtain the number of rows (row-count#), the number of columns (column-count#), the output column names (column-names#) and the actual data (column-data#).

    The example above is typical for applications that export data or move it around while massaging the data in some ways.

    Result-set data array


    Normally you'd obtain query results using query-result# as in:
    run-query#some_query="select firstName, lastName from employee"
       query-result#some_query, firstName as define first_name
       query-result#some_query, lastName as define last_name
       Employee (
       print-web first_name
       ,
       print-web last_name
       ) found<br/>
    end-query
    In some cases, it may be better to not refer to result data set in this fashion, meaning by column name. For instance, in a dynamic query you may not know the table name, nor column names. In these cases, you can use column-data# markup:
    void get_table_data (const char *table_name)
    {
       /*<
       //
       // Construct the run-time text of dynamic SQL
       //
       write-string define qry_txt
       select * from <?print-noenc table_name?>
       end-write-string

       //
       // Connect dynamic query with the run-time text of SQL
       // We use with-unknown-output to demonstrate a solution if we don't know
       // (or is difficult) to obtain output columns of a query
       //
       define-dynamic-query#get_tab = qry_txt with-unknown-output

       //
       // Run the dynamic query
       //
       run-query#get_tab
       end-query

       //
       // Obtain number of rows, number of columns, column names, the actual column data
       //
       row-count#get_tab as define row_count
       column-count#get_tab as define col_count
       column-names#get_tab as define col_names

       //
       // Get actual table data
       //
       column-data#get_tab as define col_data

       
       //
       // In a loop, go through all rows, and for each row, display all column info as well
       // as the actual column data from the table.
       //
       define-int i
       define-int j
       for j = 0; j <atol(row_count); j++
           //
           // Display columns for each row
           //
           for i = 0; i <atol(col_count); i++
               print-out "colname %s, coldata %s\n", col_names[i], col_data[j*atol(col_count)+i]
           end-for
       end-for
       >*/
    }
    In the above, a C function get_table_data prints out (to stdout) a list of columns, and a column value (note that this would have to be a command-line program since that's where we'd use print-out).

    column-data lets us obtain all table's data in a char**, i.e. in an array of strings, where all table's data is laid out in a single data array, organized by repeating rows. For instance, suppose that table name in our example has 2 columns. In that case, col_data[0] and col_data[1] would be the two columns' values from the first row, col_data[2] and col_data[3] would be the two columns' values from the second row, col_data[4] and col_data[5] would be the the columns' values from the third row etc.

    column-data# must be used with as - in other words a suitable char** variable must be provided, either defined with as define or provided as in:
    c char **col_data;
    column-data#get_tab as col_data

    Using query results more than once


    A query result set can be looped through again:
    run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
       query-result#my_query, name <br/>
    end-query

    loop-query#my_query
       Again: query-result#my_query, name <br/>
    end-query

    loop-query# will go through the entire result set again, producing this output:

    Linda
    John
    Linda
    John

    Note that loop-query# does not execute a query again, but rather only uses the already obtained results.

    Copying query text in multiple places safely (query shards)



    Re-using query text in multiple places (query fragments)


    To reuse the text of query segment, use query-fragment#:
    input-param find_name
    input-param find_year

    query-fragment#my_fragment="select name, extract(year from dateOfHire) yearOfHire from employee "

    if-string find_name!=""
       run-query#my_query_one="<?query-fragment#my_fragment?> where name=<?find_name?>"
    else-if-string find_year!=""
       run-query#my_query_one="<?query-fragment#my_fragment?> where dateOfHire > '<?find_year?>-01-01 0:0:0' "
    end-if
    Query fragment is a string constant that can be used within a query text. It is useful where shards are not practical, such as if a query fragment is too long, or used in too many places. Each query is still static because the text is known at compile time. A fragment can have other constructs in it, such as input parameters, other fragments or shards.

    Loop control in result set


    When looping through the result set, you can use continue-query and exit-query to continue at the top of the loop or exit the loop, respectively:

    This will print only the first employee name:
    c int i = 0;
    run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
       query-result#my_query, name
       c i++;
       if i>0
           exit-query
       end-if
    end-query
    This will skip the first employee name and print the rest:
    c int i = 0;
    run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
       c i++;
       if i==1
           continue-query
       end-if
       query-result#my_query, name
    end-query

    Error handling in executing queries


    For INSERT, DELETE and UPDATE statements, error value is available in the result set:
    run-query#my_query="insert into employee (name, yearOfHire) values ('Terry', 2017)"
       query-result#my_query, error as define err
       if atol(err)!=0
           report-error "Error in adding employee, error %s", err
       end-if
    end-query
    SELECT statements must succeed (with or without results). If not, it is an irrecoverable error.

    Getting auto_increment value


    To obtain auto_increment value after insert, use insert_id value:
    run-query#my_query="insert into employee (name, yearOfHire) values ('Terry', 2017)"
       query-result#my_query, insert_id as define employee_id
       if atol(err)!=0
           report-error "Error in adding employee, error %s", err
       end-if
       New employee added, with an ID of
       print-web employee_id
       <br/>
    end-query
    Just like with most other markups, you can define the string as you're getting it, or use the string you already defined:
       query-result#my_query, insert_id as define employee_id

       // or:

       define-string employee_id
       query-result#my_query, insert_id as employee_id

    NULL and result column data types


    All results from the database come as a string type (i.e. char*). NULL value is always an empty string (""). If you want to know if a value is NULL (versus an actual empty value), use a function such as ISNULL() in the database queries.

    Executing Cloudgizer program from command line

    See this chapter in API on more information about building and executing Cloudgizer program from command line.

    Executing external programs

    For most use cases, use exec-program:
    define-int st
    define-int out_var_len=2000
    define-string out_var
    define-string program_path="/usr/bin/some_program"

    exec-program program_path program-args "-flag1", arg1, "-flag2" program-status st program-output out_var program-output-length out_var_len
    This will execute program specified by program_path with arguments specified under program-args. Program arguments are separated by a comma. If a comma is a part of an argument, it must be escaped, as in \, and if double quote is a part of an argument it should be escaped as in \" as well.

    On return, the exit status will be st, and any output (stdout and stderr) will be in out_var. Note that out_var is allocated inside the execution function, and the length used to allocate it is out_var_len.

    If program-output-length is not specified, the out_var output variable is allocated with a default 256 bytes. Either way, any output in excess of what's available is truncated.

    You can also define the status of execution and the output buffer in the markup itself:
    exec-program "/usr/bin/some_prog" program-args "-flag1", arg1, "-flag2" program-status define st program-output define out_var
    If you want more complex use cases, such as redirecting file to stdin, and redirecting stdout/stderr to a file, or if you want to create a "pipe", i.e. execute two programs where output of one is the input of another use program execution functions.

    Sending emails

    Use send-email to send emails:
    define-int st
    send-email from "joe@xyz.com" to "mike@zyx.com" subject "Hi there" headers "MIME-Version: 1.0\r\nContent-Type: text/html" body "Let's meet at 10am<br/>" status st
    The sender of email is given by from clause ("joe@xyz.com") and the email recipient is given with to ("mike@zyx.com"). The subject is provided with subject clause ("Hi there"). Any headers are in headers clause (which indicate this is an HTML email), and body of the email is given with body clause.. The status clause ('st') is the status of sendmail program used to send the message. Headers and status are optional while other clauses are mandatory.

    Loading URL content (calling a web page) programmatically

    Use web-call to get the result of loading the content of a URL via GET method:
    web-call "https://mysite.com/any-url" with-response define webres with-error define weberr  with-cert "/home/dasoftver/ca.crt" cookie-jar "/home/dasoftver/cookies"
    A web response to a given URL is loaded into a string variable that is defined with a mandatory with-response clause. In this case variable webres is defined via define clause, but it could be defined prior to web-call. Other clauses are not mandatory. with-error specifies a string variable to hold an error text, if any. with-cert is used to specify the location of HTTPS certificate. You can also use with-no-cert if the certificate is not to be checked.

    cookie-jar specifies the location of a file holding cookies. Cookies are read from this file (which can be empty or non-existent to begin with) before making a web-call and any changes to cookies are reflected in this file after the call. This way, multiple calls to the same server maintain cookies the same way browser would do. Make sure the same cookie-jar file is not used across different application spaces.

    A simple use of web-call:
    define-string webres
    web-call "http://mysite.com/any-url" with-response webres
    or without checking HTTPS certificate:
    define-string weberr
    web-call "https://mysite.com/any-url" with-response define webres with-error weberr  with-no-cert
    Response that redirects to another URL will be followed up to 4 times. The response is the body message, not the headers.

    This function does not return length of a response, because it is meant to be used as a text-based (i.e. null-terminated string based) messaging system. For uploading or downloading files, a more encapsulated method would be to use command-line utilities (such as curl or rcopy) via program execution functions.

    Strings


    Strings are 'char *' variables that are allocated by Cloudgizer's memory allocation system. Typically you never free them. They are freed at the end of each request.

    If you are using any cld_* function that alters strings (i.e. as an output parameter) or in markup constructs that alter strings (such as append-string or write-string), then such strings must be defined using one of the following methods, or by means of API functions such as cld_malloc, cld_calloc or cld_realloc. Usage of other kinds of 'char *' variables will likely be detected by Cloudgizer and program halted.

    Defining and copying strings


    You should not assign NULL value to a string. Use empty strings (as described above) instead.

    Appending string

    Use append-string:
    define-string str1="Hello "
    define-string str2="world!"

    append-string str2 to str1

    print-web str1
    The output is:

    Hello world!

    Search and replace string

    Use subst-string and subst-string-all:
    define-string my_str = "Have a good day and good night!"
    subst-string "good" with "great" in my_str

    print-web my_str
    In the above example, the very first occurance of 'good' is replaced with 'great', producing the output:

    Have a great day and good night!

    If you wish to replace all occurances, use subst-string-all:
    define-string my_str = "Have a good day and good night!"
    subst-string-all "good" with "great" in my_str

    print-web my_str
    The above will produce the output:

    Have a great day and great night!

    The string where replacement takes place (in this case 'my_str') must have been defined with define-string.

    Constructing complex strings

    Instead of output being sent to the Web client, it can be directed into a string.  Use write-string and end-write-string to do this:


    All leading and trailing whitespaces are trimmed from each line. Only the whitespaces within each line are output. Text is output as non-encoded (as opposed to URL or web encoded), unless you specifically output something encoded otherwise.

    Program flow

    Loops

    Use for and end-for markups:
    define-int it
    for it=0; it < 10; it++
       Outputting #<?print-int it?><br/>
    end-for
    In the above example, a simple loop outputs this to the web page:

    Outputting #0
    Outputting #1
    Outputting #2
    Outputting #3
    Outputting #4
    Outputting #5
    Outputting #6
    Outputting #7
    Outputting #8
    Outputting #9

    Conditional statements (if, else...)

    Conditional statements can be used as follows:

    Files

    File storage


    You can store your files anywhere you like and use either standard C file mechanism to handle them or Cloudgizer API (such as reading/writing files etc.). Cloudgizer however, also provides a file storage mechanism that you can use for any general purpose. This file storage is used for automatic upload of files as well.

    In this file storage, files are generally stored in file directory under application's home directory. Your application may use this storage for any documents you create, see the API you can use to create your own files. This storage is also used for uploading of files from web clients (such as uploading files from web browsers or mobile device cameras) - the uploading is handled automatically by Cloudgizer.

    About 30,000 files will be stored in a single subdirectory (i.e. /home/user/file/d<number>) before moving to the next subdirectory. This number of files may increase in the future, however that will not affect files in existing installations, i.e. they will stay where they are.

    The number of actual subdirectories and files is in theory limited only by the filesystem used, and Cloudgizer's choice of the number of files is not a hard limit, hence it may increase in the future. The number chosen currently is reflective of the common device storage available at present and the subjective typical file size distribution across typical business applications, so clearly the choice is not related to theoretical limits or considerations.

    Modern filesystems can support great many files. For example, if you had 120 million files, they would be spread across some 40,000 directories with 30,000 files each. For example, files might be named /home/user/file/d0/f31881, /home/user/file/d10/f321214, etc.

    File names are always based on the ID number, so that changing the number of files per subdirectory will never overwrite files. This scheme also allows for faster access to file nodes due to relatively low number of files per subdirectory, and also might allow for hardware or network partitioning.

    Typically, an application will store file names in the database and actual files in file storage in file directory. Each file has a unique ID, and you can use this ID to reference a file in the database, see an example in file creation API.

    The table cldDocumentIDGenerator is used to create unique file names across the application, even when multiple requests are made at the same time. This table is created as part of Cloudgizer's installation and used internally to deliver this functionality - you don't have to ever deal with it.

    Permissions and ownership


    Any files created by Cloudgizer (i.e. through uploading from web clients, writing files etc.) are owned by Cloudgizer user and have 700 access permission (owner-only reading, writing and accessing). Group has no privileges, and neither have other users. The Cloudgizer user's home directory is also set up with 700 access permission.

    Uploading files


    Cloudgizer will upload files for you automatically. The file will be stored in file directory by using the document ID generator as a basis for subdirectory and file name under the file directory.

    For example, files uploaded might be named /home/user/file/d0/f31881, /home/user/file/d10/f321214, etc. See File storage for more details on how files are stored.

    Reading a whole file

    Use read-file to read a whole file:
    read-file "/home/user/some_file" to define file_content status define st
    if st>=0
       Read:
       <hr/>
       print-web file_content
       <hr/>
    else
       Could not read (<?print-int st?>)
    end-if
    The above example will read a file /home/user/some_file and store the result into a variable file_content and the status into variable st.  Variable st would be -1 if cannot open the file, -2 if cannot read the file, and the size of the file read (0, or a positive number) if reading succeeded.  

    define in both the content variable and the status are optional, so it could be:
    define-int st
    define-string file_content

    read-file "/home/user/some_file" to file_content status st

    Writing a whole file or appending to a file

    Use write-file to write to a file:


    Copying a file

    Use copy-file to copy a file to another file. The target file is created if it does not exist.
    copy-file "/home/user/source_file" to "/home/user/target_file" status define st
    print-int st
    This copies from file source_file to file target_file. The result is -1 if cannot open source_file, -2 if cannot open target_file, -3 if cannot read source_file, -4 if cannot write target_file, and 1 on success.

    You can omit define for status:
    define-int st

    copy-file "/home/user/source_file" to "/home/user/target_file" status st

    Web address of server

    Web address is the URL specified in config file in web_address variable.

    Use web-address to get the base URL for server. Use it for links and forms:


    Error handling in general


    If an irrecoverable error occurs, then oops function will be called. Irrecoverable error is an error that cannot be handled, it always means the end of a request. Examples of such errors are invalid SQL statements or an invalid URL request. oops function is like this:
    #include "cld.h"
    ..
    void oops(input_req *req, const char *err);
    input_req is a type that contains information relating to the current request. err is an error message itself.

    You can change the implementation of oops function from what it is by default.

    When checking errors in your application (i.e. errors that are recoverable), and if upon the examination of the error, your code decides it is still unrecoverable, do not call oops directly. Rather use report-error:
    if cannot_recover==1
       report-error "Error in processing, error %s, details %s", err, details
    end-if
    report-error takes arguments the same way as standard printf function.  

    Include files


    Do not use Cloudgizer markup code in include files. Instead place all of it in .v files, which are translated into .c files. Use inline code if you need to avoid function calls.

    Debugging your code

    Generally, you can debug Apache server by forcing it into a single-process mode, running it in gdb and hunting for problems there. A good tutorial on this can be found at https://httpd.apache.org/dev/debugging.html. Once in debugger, you'd want to break in cld_main function, which is the root function for any Cloudgizer application. If for some reason this doesn't work, you have another option, described here.

    Tracing and debugging options

    These options are located in the trace/debug file, meaning debug file in the trace directory under application's home directory. Typically, you'd have this in it during development:
    sleep=-1
    trace=1
    lint=1
    memorycheck=1
    tag=<anything you like>
    Here is the breakdown of what these do:


    Make sure not to use sleep, lint, memorycheck and tag in your production code.

    Finding where program crashed


    When your program crashes, your best bets are to look in:


    Both web-page-crash and backtrace file will attempt to show the full stack dump, and there you can see function names and line numbers (if available and if you compiled with debugging information included). Cloudgizer will analyze your executable and attempt to show the exact line in your source where the problem happened (meaning the exact line in .v code, and not in the generated .c code, alleviating the need to connect the two files!).

    For example, you'd see a report like this that exactly pinpoints the location of the crash (in this case due to segmentation fault) - you'd look for the source code just before entering signal_handler and you'll also see the entire stack leading up to invocation of the function where the problem is:
    START STACK DUMP ***********
    28014: 2018-06-01-20-54-04: Caught SIGSEGV: segmentation fault
    last known tracing file/line: [cldrt.c][2697]
    -----
    cld_get_stack
    /home/hellocld/src/chandle.c:165
    /usr/local/lib/cld/libacld.so(cld_get_stack+0x50) [0x7f4cd747016c]
    -----
    posix_print_stack_trace
    /home/hellocld/src/chandle.c:136
    /usr/local/lib/cld/libacld.so(posix_print_stack_trace+0x10) [0x7f4cd7470101]
    -----
    signal_handler
    /home/hellocld/src/chandle.c:264
    /usr/local/lib/cld/libacld.so(signal_handler+0x1f0) [0x7f4cd7470504]
    -----
    __restore_rt
    sigaction.c:?
    /lib64/libpthread.so.0(+0xf370) [0x7f4cdd086370]
    -----
    home_landlord
    /home/hellocld/rentomy/home_landlord.v:199
    /usr/lib64/httpd/modules/libcldapp_rentomy.so(+0xab200) [0x7f4cd7752200]
    -----
    home
    /home/hellocld/rentomy/home.v:155
    /usr/lib64/httpd/modules/libcldapp_rentomy.so(+0xa05f2) [0x7f4cd77475f2]
    -----
    pass_reset
    /home/hellocld/rentomy/pass_reset.v:359
    /usr/lib64/httpd/modules/libcldapp_rentomy.so(+0x9ed87) [0x7f4cd7745d87]
    -----
    cld_handle_request
    /home/hellocld/rentomy/cld_handle_request.v:291
    /usr/lib64/httpd/modules/libcldapp_rentomy.so(+0x57e6) [0x7f4cd76ac7e6]
    -----
    cld_main
    /home/hellocld/rentomy/a_cldapp.c:69
    /usr/lib64/httpd/modules/libcldapp_rentomy.so(+0x4248) [0x7f4cd76ab248]
    -----
    cld_handler
    /home/hellocld/rentomy/mod.c:52
    /usr/lib64/httpd/modules/libcldapp_rentomy.so(+0xad4d4) [0x7f4cd77544d4]

    Enabling debugging information

    To enable debugging with gdb, set CLDDEBUG variable in supplied cldmakefile to 1 when making your application. This will turn on debug code generation for gdb.

    Which shared libraries are loaded?


    Check the beginning of the request's trace file - it shows all shared objects loaded, their names and their address ranges. This is useful when figuring out the configuration of Apache server, and otherwise.

    API


    The following is the list of publicly available data types and functions.

    Do not use API for common markups provided otherwise. For example, do not use input parameters API, but rather use input-param markup. If underlying API and types change, you'll have to change your code.

    Use API only if there is no alternative. The same goes for any other C code you may write, be in direct C code or through c and start-c markups.

    Memory allocation

    Allocation and garbage collection

    The following memory allocation functions are available:
    void *cld_malloc (size_t size);
    void *cld_calloc (size_t nmemb, size_t size);
    void *cld_realloc (void *ptr, size_t size);
    void cld_free (void *ptr);
    char *cld_strdup (const char *s);
    The above have the same interface as the standard library functions without the 'cld_' prefix.

    Always use define-string to allocate memory - only if otherwise necessary, use above cld_* functions for memory allocation.

    Most all markups and API will automatically create or expand/shrink memory when needed.

    Never use cld_free unless absolutely needed, for example if your request allocates lots of memory in a loop and needs to release it in each pass during a single request.

    Memory garbage collector is provided and will release all allocated memory at the end of request.

    Do not try to minimize memory usage of a request unless you have a very good reason to, because most requests take relatively short time to complete and memory will be released shortly. Great many bugs and crashes come out of using free()-like functions improperly.

    Never use library functions malloc(), free() etc. unless you have an extremely compelling reason, for example if you need to allocate memory and some library frees it. Do not use pointers obtained from Cloudgizer with library functions such as free() or realloc(), and vice versa.

    Get size of memory allocated for variable

    When variable is allocated by means of define-string or cld_malloc (or such) functions, you can determine at run-time how many bytes are currently allocated for this variable:
    int sz;
    char *ptr = cld_malloc (200);
    cld_check_memory (ptr, &sz);
    Variable 'sz' will have value of somewhat over 200 (accounting for overhead in allocating memory). If pointer 'ptr' was not allocated by Cloudgizer functions or markup, program will error out.

    Check if allocated memory is valid, overwritten or underwritten

    If you want to check if memory pointer is valid and if memory hasn't been overwritten or underwritten, use cld_check_memory:
    int i = cld_check_memory (p, NULL);
    p is a pointer returned from define-string, cld_malloc or other such functions. This is an assertive function, i.e. your program will report an error if the memory block is invalid. If memory is okay, it returns an index into an internal memory array that holds information about allocated pointers. This index is for internal purposes only, do not use it in any way.

    Configuration parameters


    Configuration parameters are filled from config file, located in the application's home directory.

    Use members of cld_get_config()->app structure to get the following:


    Tracing and debugging

    Write to trace file


    Trace call allows you to write to current process' trace file:
    CLD_TRACE("Entering new employee, id %s", id);)
    Trace call uses printf syntax to output data.  

    Trace will be written only if trace parameter in debug file is 1.

    Which trace file is being written to?


    The name of current trace file is:
    cld_get_config()->app->trace.fname

    Using custom tag value for debugging


    Use cld_get_config()->debug.tag to access tag field in the debug file. This field can have any value you want. Use it to control some aspect of debugging process, in a conditional fashion:
    if-string cld_get_config()->debug.tag = "$debug_check$"
       ...
    end-if
    In production, you would clear the tag value.  

    Linting your HTML code dynamically

    Use cld_lint_text to check validity of HTML code generated dynamically at run-time (for example check HTML emails before sending or check snippets of generated code not directly displayed):
    write-string define html_code
       <!DOCTYPE html>
           <html>
               <body>
                    Hello<br/>
                    This is html code.<br/>
                    <hr/>
               </body>
           </html>
    end-write-string

    c cld_lint_text (html_code);
    In this example, Cloudgizer will check validity of HTML in html_code (as XHTML) by using xmllint utility. lint in debug file must be set to 1 in order for cld_lint_text to do this. If lint is 0, nothing will be done.

    If there is an error, program will stop and you can see more information in program's trace file (under trace directory).

    Note that Cloudgizer will lint your web output automatically if lint is 1. Use this function to check snippets of HTML that are not output directly by your program.

    URL input

    URL input parameters


    Input parameters are stored in
    cld_get_config()->ctx.req->ip
    Total number of input parameters is:
    cld_get_config()->ctx.req->ip.num_of_input_params
    Names and values of input parameters are:
    cld_get_config()->ctx.req->ip.names[0], cld_get_config()->ctx.req->ip.values[0]
    cld_get_config()->ctx.req->ip.names[1], cld_get_config()->ctx.req->ip.values[1]
    ...
    etc. up to the total number of input parameters

    Is URL referrer from the same site?


    Use cld_get_config()->ctx.req->from_here to determine if the current web request originated from the same site or not. For example, you may not want to spend bandwidth on images not shown from your application, if that's how it's designed. This flag is 1 if the current web request originated from the same site, otherwise 0.

    Note that from_here will be 1 if the URL is opened from a web client directly by the end-user.

    URL string that is being processed


    The URL of current request is:
    cld_get_config()->ctx.req->url
    and the referring URL is:
    cld_get_config()->ctx.req->referring_url

    URL parameters manipulation


    There is a cld_input_params data type that can hold input parameters. You can manipulate URL parameters directly by using cld_replace_input_param function with cld_get_config()->ctx.req->ip as input.

    You can use the following:


    Cookies

    Data type for storing

    Cookies passed from HTTP are in:
    cld_get_config()->ctx.req->cookies[0].data
    cld_get_config()->ctx.req->cookies[1].data
    ....
    up to the number of cookies, which is:
    cld_get_config()->ctx.req->num_of_cookies

    Cookies: get, set, delete, find



    Global data


    If you need global data in your program (data accessible to any function) and share it between different parts of your program, you can use:
    cld_get_config()->ctx.req->data
    This is a global void* pointer you can use any way you like for sharing global data unrelated to the current request data.

    Encoding and Decoding, URL encoding, Web encoding, Base64 encoding



    Numbers

    Create string from integer


    To get a new string from an integer, use cld_i2s function:
    print-web cld_i2s (55, NULL);
    The result will be:

    55

    To store the value in a string:
    define-string val

    c cld_i2s (55, &val);

    print-web val
    The result is the same.  

    Check if string is a positive integer or a number



    Execution of external programs, piping from one program to another


    You can execute external programs by using several functions. All of these functions return the status of program (typically 0 if okay, non-zero otherwise).

    Typical use cases are:



    Here are the details on these functions:


    Output data and HTTP header

    Outputting HTTP header, custom HTTP response headers


    To output HTTP header for dynamically generated web pages, use cld_output_http_header:
    input_req *req = cld_get_config()->ctx.req;

    cld_output_http_header(req);
    In most cases, you'd just output header as shown above. This means a client (such as browser) is instructed not to cache since pages are dynamically generated in a typical scenario and content type is HTML.

    If you however want to send a custom header, you can set it through header member of input_req:
    input_req *req = cld_get_config()->ctx.req;
    cld_header header;

    header.ctype = "..."; // set content type such as "text/css"
    header.cache_control = "..."; // set custom cache control, such as "max-age=10000"
    header.status_id = "..."; // set custom response code such as "302"
    header.status_text = "..."; // set custom response text such as "Found"
    // Set any other custom headers as name/value pairs in control/value array, which must not have gaps and unused values must be NULL
    // There can be maximum of CLD_MAX_HTTP_HEADER headers added through this mechanism.
    header.control[0] = "...";
    header.value[0] = "...";
    header.control[1] = "...";
    header.value[1] = "...";
    ...

    req->header = &header;
    cld_output_http_header(req);
    In the above example, custom header is constructed and output.

    Encoding data for output


    To output data, use cld_printf or cld_puts.



    Each of the above functions returns number of bytes written. Writing is typically buffered for maximum performance.

    "Outputting data" means outputting to the web client, or the string. If the above functions are used within write-string and end-write-string (or API equivalents), the output goes to the string, otherwise it goes to the web client, except if the program is in batch mode. In batch mode, there is no output produced by these functions - you have to output data yourself (to a file, stdout, stderr etc.) or by using print-out and print-error markups. In batch mode, you can still write to a string and then output the string however you see fit.  

    Output text without breaks


    If you want to output text to web client and substitute new lines with <br/>s, use:
    const char *text = "this is\nsome text\n";
    cld_print_web_show_newline (text);
    This outputs:

    this is<br/>
    some text<br/>

    Disable and Enable HTML output, status of enablement


    The output of text to web client can be enabled or disabled. By default it is enabled. In order to output a file, it has to be disabled.

    To disable output:
    cld_disable_output();
    To enable output:
    cld_enable_output();
    You would disable HTML output when you want to output a file from your program, in which case you would output HTTP headers on your own (see cld_out_file()).

    When HTML output is disabled, anything written so far into the web output buffer (that hasn't been output yet) is cleared. Also, anything output from that point onward is never written to the output buffer.

    If you need to know if html output is enabled or disabled, use cld_is_disabled_output() function. It returns 1 if HTML output is disabled, otherwise 0.

    Send files to the client and output HTTP headers for files


    Use cld_out_file to send files to the client, for example images, documents etc. You can do this to display these files or to have the client download them.

    Because the output from your program is always generated text, it is always sent to client with a note not to cache (unless you use custom headers).

    For outputting files, however, you can control HTTP headers. cld_header data type can be used to control it:
    cld_header header;

    //
    // Set the content type
    //
    header.ctype = "image/jpg"; // (or text/html or application/pdf etc.)
    Content type must be specified.
    //
    // Disposition, show or download
    //

    // Show file
    header.disp = NULL; // default, just show in client
    header.file_name = NULL; // default, just show in client, no file name to download

    // Download file
    header.disp = "attachment"; // used to download files (i.e. client will prompt for download)
    header.file_name = "document14"; // this is file name to be downloaded, used only when disp is "attachment"
    Default values disp and file_name are  NULL.
    //
    // Cache control
    //
    header.cache_control = "max-age: 3600"; // set cache-control.
    Default cache value is to cache almost forever (or rather for 53 years).
    //
    // ETag
    //
    header.etag = 1; // if 1, etag (file modification date) will be included, if 0, it won't be
    Default etag value is 1.
    //
    // Custom HTTP header fields
    //
    header.control[0] = "some_HTTP_option";
    header.value[0] = "value for some_HTTP_option";
    header.control[1] = "some_HTTP_option_1";
    header.value[1] = "value for some_HTTP_option_1";
    etc. up to CLD_MAX_HTTP_HEADER options. All options in control/value members must be continuosly present from index 0 up to whatever index is used.
    By default, all custom fields are NULL.

    An example:
    cld_header header;

    // Initialize header
    cld_init_header (&header);

    // Set header
    header.ctype = "image/jpg";

    // Output file with header
    cld_out_file ("/home/user/myfile.jpg", &header);
    Both cld_init_header and cld_out_file do not return a value.

    Note that cld_out_file will error out if your file name has dot-dot (..) in it - such files will not be served to avoid dot-dot (or path-traversal) attacks. Your file name should never have dot-dot (i.e. "..") in it.

    File not found (404)


    To output File Not Found message (i.e. 404 code back to the web client), use:
    cld_cant_find_file ("Explain reason for Not Found");
    To use this function, HTTP header must not have been output.

    Flush output


    The program output can be flushed with  cld_flush_printf:
    int num_bytes_written = cld_flush_printf(0);
    The input parameter to this function is always 0. Returns -1 if no output could be flushed (meaning HTML output is disabled and no writing to string is happening now), or number of bytes written (0 or more).
    Typically you don't need to call this function, and it may slow down the output if you do, even if your output is large, since flushing is done automatically when needed. In rare occassions you might use it, such as if your output consists of a number of parts and each part takes some time to compute.

    Create new database document and a new associated file


    Use cld_make_document:
    define-string document_id

    // do the rest in a C block
    start-c

    char document_file[MAX_FILENAME_LEN + 1];

    // Create a new document. You get:
    // 1. document_id - which is a unique ID for the document.
    // 2. document_file - a full-path file name associated with this unique ID for the document
    // You likely want to store both document_id and document_file in some table where you can reference them later.

    FILE *f = cld_make_document (&document_id, document_file, MAX_FILENAME_LEN);

    // Now you can  write to the file with file pointer f, close it, etc.
    fprintf (f, "Hello!");
    fclose(f);
    end-c

    Strings

    Initialize, Copy, Append strings


    The following functions are used for string creating and manipulation:


    Output data to string


    Program output normally goes to the web client. You can redirect it to a string by using:
    char *my_str;

    cld_write_to_string (&my_str);

    cld_printf (CLD_NOENC, "Some output that normally goes out to the client..");

    cld_write_to_string (NULL);

    int bytes_written = cld_write_to_string_length ();
    In the above example, the first call to cld_write_to_string signifies that all output will go to variable str. This is so until the next call to cld_write_to_string happens with NULL parameter. The length of the string written can be obtained immediately after the closing call with cld_write_to_string_length.

    Calls to cld_write_to_string can be nested, i.e.:
    char *my_str;

    cld_write_to_string (&my_str);

    cld_printf (CLD_NOENC, "This output goes to my_str variable..");

    char *my_str_1;

    cld_write_to_string (&my_str_1);
    cld_printf (CLD_NOENC, "This output goes to my_str_1 variable...");
    cld_write_to_string (NULL);
    int bytes_written_1 = cld_write_to_string_length ();

    cld_write_to_string (NULL);

    int bytes_written = cld_write_to_string_length ();
    In this case the second cld_printf goes to my_str_1 variable, and not to my_str variable. Variable bytes_written_1 has the length of this output only. The result for the first cld_printf is the same as before.

    cld_write_to_string can be used to capture page output:
    define-string my_str

    c cld_write_to_string (&my_str);

    Hello!<br/>
    <br/>
    It is a good day today!<br/>
    <hr/>

    c cld_write_to_string (NULL);

    print-noenc my_str
    In this above example, the output (Hello!... up to and including <hr/>) is written into my_str string, which is then output.

    Change case of strings (upper, lower)


    To lower or uppercase a string, use:
    char my_str[] = "SOME STRING";
    cld_lower(my_str);
    Variable my_str is now "some string".
    cld_upper(my_str);
    Variable my_str is now "SOME STRING".

    Both cld_lower and cld_upper return the input string, in this case they would both return my_str.

    Substrings



    Trim string


    To trim string of whitespace both on left and right, use cld_trim:
    char *str = cld_strdup (" ABC ");
    int len = strlen (str);

    cld_trim (str, &len);
    Variable str will be "ABC" and len will be 3 (i.e. the new length). This function doesn't return a value.  

    Break down string into an array of strings based on a delimiter


    Often times a string needs to be broken down into pieces based on a variable delimiter. To do that, use cld_break_down:
    define-string to_break="name=value+name1=value1+name2=value2"
    start-c
      cld_broken broken;
      cld_break_down (to_break, "+", &broken);
      int i;

      for (i = 0; i < broken.num_pieces; i++)
      {
           cld_broken broken_further;
           cld_break_down (broken.pieces[i], "=", &broken_further);
           cld_printf (CLD_WEB, "Name is %s, value is %s", broken_further.pieces[0], broken_further.pieces[1]);
           cld_printf (CLD_NOENC, "<br/>\n");
      }
    end-c
    The output would be:

    Name is name, value is value
    Name is name1, value is value1
    Name is name2, value is value2

    The string to_break is broken by inserting zero characters, for better performance. If you want it unchanged, use cld_strdup to make a copy first, and then break down the copy. The variable 'broken' (of cld_broken type) has num_pieces member (the number of pieces broken down to) and pieces (the array of pieces).  

    In the above example, we first break down the original string into pieces separated by "+", and then break those down separated by "=" (in general the separator is any string). Since the original string is broken down, you can also traverse the pieces by reading them zero-terminated one by one from the original string to_break.

    This is an efficient way of breaking down a string without making any copies.

    Clear Unused variable error


    It is a good practice to delete unused variables, but sometimes they cannot be. To prevent compiler warnings, use:
    CLD_UNUSED(x);

    Error reporting


    Fatal errors (resulting in program termination) can be reported by cld_report_error which takes the arguments the same as printf and returns number of bytes written:
    cld_report_error("Error %s happened, terminating", err);
    To report errors and do not exit, use:
    cld_report_error_no_exit("Error %s happened, continuing", err)
    In either case, any currently open database transactions are rolled back.

    Encryption and hashing

    Cloudgizer uses 256 bit SHA hashing and AES encryption. The cipher used is AES-256-CBC.

    Hashing


    To create a 256-bit SHA hash, use cld_sha:
    char *hash = cld_sha("some value to hash");
    Variable hash now contains hashed value of "some value to hash". This is a string terminated by null character, always 64 bytes in length. Hashed value might look something like 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'.

    Encrypting data


    Use cld_aes_encrypt, for example:
    char *to_encrypt = "some data to encrypt";
    int len = strlen(to_encrypt);
    EVP_CIPHER_CTX e_ctx;
    cld_get_enc_key("password", "salt", &e_ctx, NULL);
    char *encrypted_data = cld_aes_encrypt(&e_ctx, (unsigned char*)to_encrypt, &len, 0);
    The output of the function ('encrypted_data') contains encrypted data, which is allocated by Cloudgizer.

    In  the above example, we first create an encryption context out of password and salt ("password" and "salt" in the above example) and encrypt string 'to_encrypt' of length 'len'. The fourth parameter is 0 (character mode of encryption), meaning the result is the character string (i.e. the encrypted value is null-terminated and suitable for using in database varchar columns, for example).  If the fourth parameter is 1 (binary mode of encryption), the result is a binary encrypted string.

    If it is character mode, then the encrypted output will be converted into a string of hexadecimal characters (i.e. ranging from '0' to '9' and from 'a' to 'f') terminated by null character. The maximum size of a buffer of encrypted string is 2*(length_of_input_string+16)+1. This accounts for the maximum of 16 bytes extra for AES, and double for hexadecimal representation (in case of character mode of encryption).  Character mode of encryption is convenient if the result of encryption should be a human readable string, or for the purposes of non-binary storage in the database.

    Variable 'len' holds the length (in bytes) of data to encrypt, and on return it holds the length (in bytes) of the encrypted data (excluding zero byte at the end if the fourth parameter is 0, i.e. if the result is a character string).

    The salt can be NULL in which case no salt is used.

    Decrypting data


    Use cld_aes_decrypt. If 'encrypted_data' is encrypted data produced by cld_aes_encrypt:
    EVP_CIPHER_CTX d_ctx;
    cld_get_enc_key("password", "salt", NULL, &d_ctx);
    int len = strlen (encrypted_data);
    char *decrypted_data = cld_aes_decrypt(&d_ctx, (unsigned char*)encrypted_data, &len, 0);
    The output of the function ('decrypted_data') contains decrypted data, which is allocated by Cloudgizer.

    In  the above example, we first create a decryption context out of password and salt ("password" and "salt" in the above example) and decrypt string 'encrypted_data' of length 'len'. The fourth parameter is 0, meaning that 'encrypted_data' is a character string produced by cld_aes_encrypt with fourth parameter of 0. When decrypting, password, salt and the mode of encryption (binary or character) must match between cld_aes_encrypt and cld_aes_decrypt.

    Variable 'len' holds the length (in bytes) of encrypted data (excluding zero byte at the end in case of character mode of encryption), and on return it holds the length (in bytes) of the decrypted data (excluding zero byte at the end which is always placed at the end of decrypted data).

    Generating random string


    To generate a random string (null-terminated) use cld_make_random(). For example, to generate a random string, use:
    char rnd[20];
    cld_make_random (rnd, sizeof(rnd));
    Variable 'rnd' now holds random string of length 19. The buffer 'rnd' must be at least a byte longer than the length of random string you desire, to accomodate null terminator.

    Storing and retrieving data in a sequential list (FIFO)


    To store data in a sequential list, and be able to rewind to the beginning and retrieve later, use functions related to cld_store_data type:
    // Declare list data
    cld_store_data list_data;

    // Initialize list data
    cld_store_init (&list_data);

    // Store name/value pairs to list data
    cld_store (&list_data, "item name 1", "item value 1");
    cld_store (&list_data, "item name 2", "item value 2");
    cld_store (&list_data, "item name 3", "item value 3");
    ...

    // Rewind list data to the beginning
    cld_rewind (&list_data);

    // Retrieve name/value pairs in the order in which they were put in
    char *item_name;
    char *item_value;
    while (1)
    {
       cld_retrieve (&list_data, &item_name, &item_value);
       if (name == NULL) break;
       //
       // item_name and item_value will be "item name 1"/"item value 1", "item name 2"/"item value 2" etc.
       //
    }

    // Delete list data
    cld_purge (&list_data);

    Executing Cloudgizer program from command line (Batch processing)

    Using batch processing


    Batch processing is using the program as a command line program. To enable batch processing, use cld_enable_batch_processing:
    cld_enable_batch_processing();
    Batch processing disables web output (HTML output). Only output to a string will work, while web output will not. You can output to stdout and stderr (either via C functions like printf or via markups like print-out and print-error), and read from stdin. A batch program can still receive input parameters via URL, through environment variables, much like web server would:
    export REQUEST_METHOD=GET
    export QUERY_STRING="page=home&action=employees"
    export CLD_BATCH_PROCESSING=yes
    ./your_program
    When writing QUERY_STRING, make sure that actual data is URL encoded. For that, use -urlencode option for cld command line utility, like in this example:
    export REQUEST_METHOD=GET
    export HELLO_WORLD_URL_ENCODED=$(cld -urlencode "hello world")
    export QUERY_STRING="page=key&dt=${HELLO_WORLD_URL_ENCODED}&action=file"
    export CLD_BATCH_PROCESSING=yes
    ./your_program
    cld -urlencode "hello world" will output URL encoded "hello world" string, with space substituted with '%20'. This output is stored into an environment variable, and its contents then used in QUERY_STRING as URL-encoded string for a URL passed to your program.

    In the above bash example, REQUEST_METHOD is set to GET, and QUERY_STRING is set to request's input parameters in URL notation. You must set CLD_BATCH_PROCESSING to yes, and examine it in your program:
    ...
    const char *batch_env = cld_ctx_getenv ("CLD_BATCH_PROCESSING");
    if (!strcasecmp (batch_env, "yes"))
    {
       cld_enable_batch_processing();
       /*<
       input-param action
       
       if-string action="employees"
           write-string define employees
               List of employees
               <hr/>
               run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
                   query-result#my_query, name as define name
                   query-result#my_query, yearOfHire as define year_of_hire
                   print-web name
                   ,
                   print-web year_of_hire
                   <br/>
               end-query
               <hr/>
           end-write-string
           print-out "%s", employees
       end-if
       >*/
       return;
    }
    ...
    In this program, cld_ctx_getenv is first used to enable batch processing with cld_enable_batch_processing, meaning there is no web output. However, you can use most all other features such as working with input parameters, writing strings, queries etc. In the end, the program outputs to standard output:

    List of employees
    <hr/>
    Linda,2001<br/>
    John,2003<br/>
    <hr/>

    Batch programs can be often written together with web programs if their business logic is intertwined. Effectively you can write programs that could be used as both web and command-line programs, for example:
    ...
    const char *batch_env = cld_ctx_getenv ("CLD_BATCH_PROCESSING");
    int is_batch = !strcasecmp (batch_env, "yes"));

    if (is_batch)
    {
       cld_enable_batch_processing();
    }
    else
    {
       /*< output-http-header >*/
    }

    /*<
    input-param action

    if-string action="employees"
       write-string define employees
           List of employees
           <hr/>
           run-query#my_query="select name, extract(year from dateOfHire) yearOfHire from employee"
               query-result#my_query, name as define name
               query-result#my_query, yearOfHire as define year_of_hire
               print-web name
               ,
               print-web year_of_hire
               <br/>
           end-query
           <hr/>
       end-write-string
       if is_batch
           print-out " %s\n", employees
       else
           print-noenc employees
       end-if
    end-if
    >*/
    ...
    The above program can be called both from the web client to output a list of employees, and it can output this list to standard output. The URL parameters used are the same. Batch program's output would be the same as above, while the output to web client would be (for URL such as http://myserver.com/go.your_app_name?page=home&action=employees):

    List of employees
    Linda,2001
    John,2003

    Exit code for batch processing


    To set exit code for a command-line program, use:
    cld_set_exit_code(3);
    This sets exit code to 3.

    POST URL (web call)


    To perform a web call via HTTP/HTTPS:
    char *response;
    char *error;
    char *cert;
    int result = cld_post_url_with_response ("https://somesite.com?par=3", &response, &error, &cert);
    The function returns 1 on success and 0 on failure. Variable error has error text in case of failure. If HTTPS is used the location of certificate file can be provided in variable cert. If cert is NULL, certificate is not checked. The actual response (minus headers) is stored in variable response. Response that redirects to another URL will be followed up to 4 times. The response is the body message, not the headers.

    Files

    File storage


    You can read and write any files accessible to you, using either standard C mechanisms, or the file API explained in this documentation.

    You can also use a provided file storage under 'file' directory - see more about this file storage.

    To create a file in this file storage, use the following code:
    define-string document_id
    c char document_file[MAX_FILENAME_SIZE];
    c FILE *f = cld_make_document (&document_id, document_file, MAX_FILENAME_SIZE);
    Output variables are document_id (containing the unique ID of the file created, produced by using cldDocumentGenerator table), document_file (containing unique file name with a complete path to it), and file descriptor 'f' as the file created is opened and ready to write to.

    You'd typically store file ID and its path in the database, and use them to later manipulate the file (read, write, delete etc.).

    Get file size


    To get the size of a file:
    size_t sz = cld_get_file_size ("/home/user/my_file");
    The return value will be -1 if cannot open file or its size.

    Copy files

    int result = cld_copy_file ("/home/user/from_file", "/home/user/to_file");
    This copies from file from_file to file to_file. The result is -1 if cannot open from_file, -2 if cannot open to_file, -3 if cannot read from_file, -4 if cannot write to_file, and 1 on success.

    Read entire file into a string variable

    char *file_data;
    int result = cld_read_whole_file ("/home/path/my_file", &file_data);
    This reads file my_file into string variable file_data, which is cld_malloc() allocated (like all Cloudgizer memory is). Variable 'result' is -1 if cannot open my_file, -2 if cannot read my_file, and the size of the file read (0, or a positive number) if reading succeeded.  

    Write whole file from a string

    char content[] = "some content";
    size_t content_len = strlen(content);

    int result = cld_write_file ("/home/path/my_file", content, content_len, 0);
    This writes "some content" to my_file. Variable 'result' is -1 if cannot open my_file, -2 if cannot write it, and 1 if writing succeeded. The whole file is overwritten with the new content.

    Append to file from a string

    char content[] = "new content";
    size_t content_len = strlen(content);

    int result = cld_write_file ("/home/path/my_file", content, content_len, 1);
    This appends "new content" to the end of my_file. Variable 'result' is -1 if cannot open my_file, -2 if cannot write it, and 1 if writing succeeded.

    Lock a file


    You can create a file and lock it so your process has exclusive read and write lock on it. This can be useful as a process control means, for example to ensure only one process can continue work.
    // returns 0 if cannot lock, -1 if cannot open file, 1 if locked,-2 invalid path
    int file_descriptor;
    int result = cld_lockfile ("/home/user/file_to_create", &file_descriptor);
    The function returns -2 if file path is invalid, -1 if cannot open file, 0 if cannot lock, and 1 if locked.

    The process must not close stdout, stderr or stdin while file lock is in effect.

    The file stays locked until the program ends, or until it's closed, for instance:
    close (file_descriptor);

    Get application home directory


    Application's home directory is a directory named after application name located directly under the home directory of the user who executes the program:
    char *home_directory = cld_home_dir ();
    If cannot determine application home directory, empty string ("") is returned.

    Application's home directory is based on application name, which you can obtain via cld_app_name().

    Check if directory


    To determine if something is a directory:
    int is_dir = cld_is_directory ("/home/user/something");
    Variable is_dir will be 1 if something is a directory, 0 otherwise.  

    Temporary files

    To store temporary files, use temporary directory:
    print-web cld_get_config()->app->tmp_directory

    Send email


    To send email via sendmail, use:
    const char *from = "addr1@something.com"
    const char *to = "addr2@somethingelse.com"
    int result = cld_sendmail (from, to, "subject of message", NULL, "This is the body of email message");
    Variable 'result' is the exit code of sendmail program. In the previous example, 4th parameter was NULL, meaning no additional header was added to the email. Email sender is 'from' and recipient 'to', followed by the subject of the message, while the message itself is the 5th parameter.

    If you want to add custom headers to the email sent (for example to send HTML email, for attachments etc.), you can add them:
    const char *from = "addr1@something.com"
    const char *to = "addr2@somethingelse.com"
    cons char *headers= "MIME-Version: 1.0\r\nContent-Type: text/html";
    int result = cld_sendmail (from, to, "subject of message", headers, "This is the body of email message");

    Queries and Transactions

    SELECT from the database

    int nrow;
    int ncol;
    char **col_names;
    char **data;

    // To select data
    cld_select_table ("SELECT ...", &nrow, &ncol, &colnames, &data);
    'nrow' is the number of rows returned. 'ncol' is the number of columns. 'colnames' is a list of column names returned. 'data' holds all column data in order in which it is retrieved (first all columns from the first row, then from the second etc.), for example if there are two columns returned then data[0] is the first column of the first row, data[3] is the second column of the second row etc.

    'data' can be NULL, while 'colnames' cannot be NULL. Maximum  number of output columns is 1000.

    To get column names only, without the data:
    cld_select_table ("SELECT ...", &nrow, &ncol, &col_names, NULL);
    In this case, col_names[0] is the name of the first column, col_names[1] is the name of the second column and so forth.

    This function does not return value, and its failure will stop the program (i.e. it is expected to always succeed).  

    Get value from auto-increment column


    Insert into a table with auto increment column produces new value for it. To obtain it:
    char val[100];
    cld_get_insert_id (val, sizeof(val));
    'val' is the output buffer. This function must be called immediately after INSERT and it does not return a value.

    Execute any SQL other than SELECT, including DDL statements

    int rows;
    unsigned int er;
    char *err_message;
    if cld_execute_SQL ("INSERT INTO ... ",  &rows, &er, &err_message)==1
       print-web "Success"
    else
       print-web "Failure"
    end-if
    Varable 'rows' is the number of rows affected, 'er' is the error number, and 'err_message' is the error message. If failed, the function returns 0, otherwise 1.

    Use this to execute any SQL, including DDL (Data Definition Language) such as CREATE TABLE, DROP TABLE etc.

    Check for open transaction

    if cld_check_transaction(1)==1
       print-web "You have an open transaction!"
    end-if
    If input parameter is 1, the function returns 1 if there is an open transaction and 0 if not. If input parameter is 0 then it will report an error if there is an open transaction and it will return 0 if not.

    If input parameter is 2, and there is an open transaction, rollback this transaction and return 0.

    Get miscellaneous

    Get web address for forms and links


    Web address (for forms and links) is obtained from config file under web_address field. You can get it from:
    const char *web_address = cld_web_address ();

    Get Base URL of server


    Base URL for "https://myserver.com/go.your_app_name?par1=val1" is "myserver.com". To obtain it, use cld_web_name and pass any URL leading to the server:
    char *base_URL = cld_web_name(cld_web_address());
    This will produce the base name of a web server running the program.

    Get version


    The software version can be obtained via following call:
    const char *major_version = cld_major_version();

    Get current, past or future time, including GMT

    To get current, past or future time, suitable for cookies, use cld_time:
    char *cld_time (const char *timezone, int year, int month, int day, int hour, int min, int sec);
    Returns time (now, in the future, or the past). Input parameter 'timezone' is the name of TZ (timezone). So to get GMT time then timezone should be "GMT", if it is Mountain Standard then it is "MST" etc.

    Input parameters year,month,day,hour,min,sec are the time to add to current time (they can be negative too). So for example ..(0,0,1,0,0,1) adds 1 day and 1 second to the current time, while ..(0,-1,0,0,0) is one month in the past.

    Get current time


    To get current time, per local time zone, use cld_current_time. The arguments are the buffer where the time should be stored and its length:
    char buf[50];
    cld_current_time (buf, sizeof(buf));
    The time returned shows year, month, day of month, hour, minute, second, separated by a dash, such as '2017-07-30-18:48:13', with hours in 0-24 range.

    Get application name


    Application name can be obtained via  cld_app_name function call, this is the same as CLD_APP_NAME in appinfo file during installation.
    /*<
     Application name is <?print-web cld_app_name()?>
    >*/
    Application's home directory is based on it. Always use cld_home_dir() to get the application home directory.

    Get environment variable (web or command shell)


    Environment variables can be available from the operating system or the web server. If the program is running in batch mode (i.e. command line), the environment used is that of the OS. When a program is running within a web server, first web server's variables are searched for a match, and if not found, the OS environment is used. The call that does this is cld_ctx_getenv:
    const char *env_value = cld_ctx_getenv ("CONTENT_TYPE");

    Copyright (c) DaSoftver LLC 2017. Cloudgizer is a trademark of Zigguro LLC. Contact email address admin@dasoftver.org. The DaSofver software and information herein are provided "AS IS" and without any warranties or guarantees of any kind.