Send GMail From the Windows Command-Line with MailWrench

Originally published on: Sun, 13 Feb 2011.

Google’s GMail service allows e-mail to be sent using an SMTPS ( SMTP over SSL ) interface. Sending GMail via the Windows command-line requires a command-line mailer with SMTPS support. My commercial MailWrench utility is one such program.

Update 25 July, 2012: MailWrench is now free, open source software. Please see the post at the following link:

https://lawlessguy.wordpress.com/2012/07/24/mailwrench-csharp-command-line-mailer-for-windows-is-now-free-software/

The GMail SMTPS service listens at port 587 on the server smtp.gmail.com. To configure MailWrench to send mail using GMail, create a text file that we’ll just name m.cfg and add the following configuration lines:

-ssl
-smtp smtp.gmail.com
-port 587
-id your_id@gmail.com
-password your_password

The MailWrench command-line you would then use to send a simple message would be something like the following:

MailWrench @m.cfg -to some_id@someserver.kom -msg "Hello!" -s "Testing!"

Note that any of the options specified in the file m.cfg could have been specified on the command-line itself. The “.cfg” files are intended to be used to contain options that rarely change when sending e-mails.

Since GMail does not allow arbitrary attachments, any file you attach must be one of the permissible attachment types ( word-processor documents, spreadsheets, images,audio, …etc. ) Your e-mail may not process properly if you send other file types.

To attach the image file test.png to the above e-mail, add the parameters -a test.png to the above command-line. The new command-line should look like the folllowing:

MailWrench @m.cfg -to some_id@someserver.kom -msg "Hello!" -s "Testing!" -a test.png

A Simple Associative Array Library in C

Originally published on: Thu, 19 Nov 2009 02:35:02 +0000

Please note: I just updated the Github sources (26 Sep 2018).  Apologies to those who tried to make that intermediate release work.

I often need to use associative arrays when writing C code. You might be more familiar with these constructs under other names ( dictionaries, maps, hashmaps, hashes, …etc. ) I will use the term map in place of the words associative array.

A map is a collection of key / value pairs or name / value pairs. My implementation of a map structure allows strings as the only data that can be stored in the name and value.

Many implementations of maps use a hashed lookup to speed up execution. I have opted to use a simple forward-chained linked-list of map structures. When searching for an item in a map, the search will sequentially pass over every item in the list of map entries until a match is found. If no match is found, an empty string is returned.

This implementation is complete enough for my needs. See:

https://github.com/jimlawless/map_lib

The test program map_stuff.c loads and displays a map with name/value pairs. The names are all numbers. The values are words that sound like the numbers ( homonyms ).

The output from the map_stuff program is as follows:

Two Too
Four Fore
One Won

Two To
Four Fore
One Won

Eight 

Eight Ate

I did not provide any kind of iterator functions in the library. If I need to iterate through a given map, I will just use a for-loop and will use the ->nxt element to reach each successive entry.

Cheating the LZW

Originally published on: Fri, 24 Apr 2009 00:10:27 +0000

In the early days of the web, many web pages sported hit-counters that would generate a graphical meter indicating the number of visitors who had viewed the page.

I thought that it would be a nice thing to have on my own site, so I set out to build my own.

I had decided to try using Perl to build my counter as it had become my language of choice when developing other CGI’s.

Some of the popular hit-counters would accept libraries of JPEG’s or GIF’s representing various kinds of digits and would then use the GD graphics library to fuse the appropriate digit images together into a single image.

I opted to take a different route.

I had decided to use the GIF format for the output image. I wanted a lossless format that could represent a small, monochrome bitmap containing six eight-by-sixteen-pixel characters.

If one has a binary character-set available, building a raw bitmap of a stream of digits is relatively painless. A monochrome bitmap eight by sixteen by two (colors) by 6 (characters) can be represented in 768 bits ( 96 bytes ) uncompressed. That’s pretty tiny!

Unfortunately, the GIF format did not permit a flag to indicate that the bitmap image would be an uncompressed bitmap. So, I was going to have to employ the LZW compression algorithm on this tiny blob of data. …or WAS I ?

The LZW algorithm uses a protocol of sorts that must be observed by both the compressor and the decompressor. An initial code-size must be agreed upon by both prior to compression or decompression. This code-size value indicates the bit-width for each code in the I/O stream. ( Like many compression algorithms, the LZW leverages variable-length bit I/O. )

The code-size must allow for two codes; clear and end-of-input.

I decided on a code-size of three bits and allowed the LZW special characters to push the code-size up to four bits … which is easier to handle in bit-shift operations. The clear-code would become eight and the EOI code would be nine as three bits can represent the numbers zero through seven.

For each bit in the generated bitmap, I wrote both a nybble with a value representing a single pixel, …a one or a zero followed by a nybble representing the clear-code.

The bit-image was emitted without any true compression. In fact, using the LZW as a protocol, the stream of raw data grew in size ( four bits to represent a monochrome pixel instead of one plus an extra four bits for each clear code. ) This process uses a byte to represent a single pixel when both nybbles are coupled.

At the end of stream, I needed to emit the EOI character, so I simple wrote a byte with the value 9*16, shifting the nine four pixels to the left so that the decompressor would stop processing the bitmap.

The Perl code below illustrates this technique by creating a GIF file named number.gif with the digits “123456” represented as a bitmapped image.

Sample digits

You might change the value of $count to something other than “123456”, but it must be six characters in length and must be comprised of all digits.

The generated image is always 805 bytes in size. 768 bytes for the bitmap, one for the EOI code, and thirty-six bytes for the GIF header and footer.

# License: MIT / X11
# Copyright (c) 2009 by James K. Lawless
#
# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation
# files (the "Software"), to deal in the Software without
# restriction, including without limitation the rights to use,
# copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following
# conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
# OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
# OTHER DEALINGS IN THE SOFTWARE.
@charset=(0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,
56,24,124,124,12,254,56,254,124,124,
108,56,198,198,28,192,96,198,198,198,
198,120,6,6,60,192,192,6,198,198,
198,24,12,6,108,192,192,6,198,198,
214,24,24,60,204,252,252,12,124,126,
214,24,48,6,254,6,198,24,198,6,
198,24,96,6,12,6,198,48,198,6,
198,24,192,6,12,6,198,48,198,6,
108,24,198,198,12,198,198,48,198,12,
56,126,254,124,30,124,124,48,124,120,
0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0);

# "clear" code
$clr=8;

# end-of-input code in left-nybble of byte
$eoi=9*16;

# must be 6-digits
$count="123456";

open(FIL,">number.gif");
binmode(FIL);
# GIF 87 header
print FIL "GIF87a" ;
print FIL pack("CC",48,0); # Raster width 48
print FIL pack("CC",16,0); # Raster height 16

# Colormap, numbr bits color res -1, 0, bits per pixel - 1
# 1 000 0 000
print FIL pack("C",128);

# Background color index
print FIL pack("C",0);
# dummy byte
print FIL pack("C",0);

# Color Map
# Black
print FIL pack("CCC",0,0,0);

# White
print FIL pack("CCC",255,255,255);

# Image descriptor
# Separator
print FIL ',' ;

# Left
print FIL pack("CC",0,0);

# Top
print FIL pack("CC",0,0);

# Width
print FIL pack("CC",48,0);

# Height
print FIL pack("CC",16,0);

# Use global color map=0, sequential image = 0, 000, pixel -1 bpp
# 0 0 000 000
print FIL pack("C",0);

# Raster data
# Code size 3 ( LZW will use 4), this makes clear code 8!, EOI 9
print FIL pack("C",3);

# here's where we'll need to send the data out in blocks. It will be

$blkcount=0; # total blocks so far
$blkinprog=0; # block in progress

for($j=0;$j<16;$j++) {
  for($i=0;$i<6;$i++) {
    for($bits=128;$bits>=1;$bits=$bits/2) {
      # Determine bit settings
      $b=substr($count,$i,1)-'0';
      if( ($charset[$j*10+$b] & $bits) == $bits ) {
        $bitout = 16;
      }
      else {
        $bitout = 0;
      }
      if( $blkinprog == 0 ) {
        if($blkcount<3) {
          $blkinprog=240;
        }
        else {
          $blkinprog=49;
        }
        print FIL pack("C",$blkinprog);
        $blkcount++;
      }
      print FIL pack("C",$bitout+$clr);
      $blkinprog--;
    }
  }
}
print FIL pack("C",$eoi);

# Zero length for block
print FIL pack("C",0);
# Terminator
print FIL ";";

close(FIL);

A Music Macro Language Player in C

I saw a reference to Music Macro Language on the HackerNews site this morning.

https://en.wikipedia.org/wiki/Music_Macro_Language

I remember embedding music strings like this in old BASIC programs. Lacking a way to tinker with MML, I first downloaded the free Macrotune libraries from:

http://www.posemotion.com/macrotune/

I then wrote a small script interpreter, mmlplayer.exe, for 32-bit Windows using MinGW. I had to correct the psg.h file by closing the top comment or I ended up seeing an #endif without #if error.

Here’s the code:

// MMLPlayer
// by Jim Lawless
// MIT / X11 License
// See: http://www.mailsend-online.com/license2015.php


#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <windows.h>
#include "psg.h"

void runscript(char *scriptname);

int main(int argc,char **argv) {
   printf("Music Macro Language Player\n");
   printf("by Jim Lawless - http://jiml.us\n\n");
   printf("This program uses the Macrotune libraries from\n");
   printf("http://www.posemotion.com/macrotune/\n\n");

   if(argc<2) {
      fprintf(stderr,"Syntax: mmlplayer script-file-name");
      return 1;
   }   
   runscript(argv[1]);
   return 0;
}

void runscript(char *scriptname) {
   char buff[1024];
   FILE *fp;
   char *p;

   fp=fopen(scriptname,"r");
   if(fp==NULL) {
      fprintf(stderr,"Cannot open script %s\n",scriptname);
      return;
   }
   while(fgets(buff,sizeof(buff)-1,fp)!=NULL) {
      p=strtok(buff,"\t\r\n ");
      if(!strcmp(p,"#")) {
         continue; 
      }
      else
      if(!strcasecmp(p,"echo")) {
         p=strtok(NULL,"\r\n");
         printf("%s\n",p);
      }
      else
      if(!strcasecmp(p,"volume")) {
         p=strtok(NULL,"\t\r\n");
         PSG_SetMasterVolume((float)atof(p));         
      }
      else
      if(!strcasecmp(p,"play")) {
         p=strtok(NULL,"\t\r\n");
         PSG_PlayMML(p,"","","");         
         while(PSG_IsPlayingMML())
            Sleep(500); // sleep and wait
      }
   }   
   fclose(fp);
   PSG_Terminate();
}

I used the following compile options after renaming psg_win_x86.lib to psg_win_x86.a:

gcc mmlplayer.c -L. -lpsg_win_x86 -o mmlplayer.exe

The script commands are as follows:

#

The # command indicates a comment. A space is required after the character.

echo string

The echo command displays the specified string on the console. It then displays a newline character.

play string

The play command plays a string of MML music.

volume float-number
The volume command sets the volume for the played strings.

Here’s a sample script that just plays a simple scale in eighth notes.

echo See MML documentation at
echo https://en.wikipedia.org/wiki/Music_Macro_Language
volume 5.0
play C8D8E8F8G8A8B8>C8
echo All done!

The Win32 package including the 32-bit DLL, mmlplayer.c, mmlplayer.exe, and the sample script can be found at:

http://www.mailsend-online.com/u/mmlplayer.zip

According to the API, it looks like one can play up to four channels of music concurrently. In the code above, I provide for only one channel of music at this time.

There’s an OS/X version of the Macrotune libraries. I might tinker with those later.

 

The 300 BAUD BBS Experience

One of the most enjoyable eras in my history of personal computing was the stretch of time from 1984 to about 1986 when the local Bulletin Board System (BBS) scene was in its infancy.

I wrote about this in another post:

http://www.mailsend-online.com/blog/bbs-fun-in-the-eighties.html

I had been trying to figure out a way to demonstrate the experience of using a 300 BAUD terminal program to others. The data transit speed on the Internet dwarfs the lowly 30 characters-per-second rate that most of us had to endure.

A couple of additional caveats often introduced more delays into the experience. On some systems such as the Commodore 64, disk I/O used a serial communication mechanism, so it was slow. In addition, many BBS’s whose core code was written in BASIC would endure pauses as the internal garbage-collect routines would sift through all of the disposed string data baggage to make room for more intermediate strings.

I was finally able to simulate the BBS experience of yesteryear by using a Commodore 64 emulator (WinVICE), Jim Brain’s TCPSer software which fools the emulator into thinking that its a smart-modem connected via the user port, Leif Bloomquist’s “Jamming Signal” BBS running on an actual Commodore 64 connected to the Internet, and Chris Dunn’s freeware CBTerm terminal software.

I first ran TCPSer with the following command-line:

tcpser -v 25232 -s 300 -p 6400 -l 7

Then, I ran WinVice with a command-line that would connect to the running TCPSer interface for simulated serial I/O through the user port:

x64.exe -acia1 -rsdev1 127.0.0.1:25232 -rsuser -rsuserbaud 300 -rsuserdev 0

Once running the emulator, I loaded up CBTerm. I then issued what appears to be a Hayes-compatible “Attention/Dial/Tone” command (ATDT) except that the “phone number” is the IP address where Leif’s BBS is located.

Here’s the entire session ( I just noticed that the Windows mouse cursor is present during the bulk of the session.  Sorry about that! ):

https://youtu.be/uGqxXpyg0BM

You’ll note that there are pauses at junctures where disk-access is likely. There are also pauses in input which could be due to garbage-collection operations.

Believe it or not, my friends and I used to do this sort of thing all the time back in the mid-80’s. A number of us used BBS’s into the 90’s, but they started to have kind of a commercial flavor. It didn’t seem much like the hobbyist community of just a few years past.

Pretty-Printing an S-Expression in Go

My friend Mike and I seem to implement Forth-like and Lisp-like mini-languages continuously.  While I used to implement small interpreters for these languages in C, I found that languages with built-in support for regular-expressions allowed me to quickly write lexical scanners for each.

I modified a set of regex’s that I had used in a Forth-like interpreter in Go ( see: https://lawlessguy.wordpress.com/2013/07/20/an-rpn-interpreter-in-go-golang/ ) so that I could parse parentheses, quoted strings, double-quoted strings, numbers (integers), and what I refer to as symbols.  Here’s the pretty-printer, sep.go, that I wrote ( “pretty” means that it indents when it encounters a left-parenthesis and removes indentation after it encounters a right-parenthesis … ) :

// Copyright 2013 - by Jim Lawless
// License: MIT / X11
// See: http://www.mailsend-online.com/license2013.php
//
// Bear with me ... I'm a Go noob.
//
// Parse an S-Expression using regexes in Go

package main

import (
	"fmt"
	"io/ioutil"
	"log"
	"os"
	"regexp"
)

var patDquote = "[\"][^\"]*[\"]"
var patSquote = "['][^']*[']"
var patNumber = "[-]?\\d+"
var patParen = "[()]"
var patSymbol = "\\S+"

var reDquote, reSquote *regexp.Regexp
var reNumber, reParen *regexp.Regexp
var reSymbol, reAll *regexp.Regexp

func indent(characters int) {
	for ; characters > 0; characters-- {
		fmt.Print(" ")
	}
}

func parse() {
	var s string
	var indent_ct int = 0

	b, err := ioutil.ReadAll(os.Stdin)
	if err != nil {
		log.Fatal(err)
	}
	input := string(b)
	tokens := reAll.FindAllString(input, -1)
	for i := 0; i < len(tokens); i++ {
		s = tokens[i]
		indent(indent_ct)
		switch {
		case reDquote.MatchString(s):
			fmt.Printf("Double-Quote String: %s\n", s)
		case reSquote.MatchString(s):
			fmt.Printf("Single-Quote String: %s\n", s)
		case reNumber.MatchString(s):
			fmt.Printf("Number: %s\n", s)
		case reParen.MatchString(s):
			fmt.Printf("Parenthesis: %s\n", s)
			if s == "(" {
				indent_ct += 2
			}
			if s == ")" {
				indent_ct -= 2
			}
		case reSymbol.MatchString(s):
			fmt.Printf("Symbol: %s\n", s)
		default:
			fmt.Printf("Unknown token: %s\n", s)
		}
	}
}

func main() {
	reDquote, _ = regexp.Compile(patDquote)
	reSquote, _ = regexp.Compile(patSquote)
	reNumber, _ = regexp.Compile(patNumber)
	reParen, _ = regexp.Compile(patParen)
	reSymbol, _ = regexp.Compile(patSymbol)

	reAll, _ = regexp.Compile(patParen + "|" + patDquote + "|" + patSquote + "|" +
		patNumber + "|" + patSymbol)

	parse()
}

Let’s assume we have a file named “tmp.txt” which contains the following:

( this is "a test" )

(add (multiply 4 5) 6)

When you redirect tmp.txt’s contents into the sep program, here’s what you should see:

sep < tmp.txt

Parenthesis: (
  Symbol: this
  Symbol: is
  Double-Quote String: "a test"
  Parenthesis: )
Parenthesis: (
  Symbol: add
  Parenthesis: (
    Symbol: multiply
    Number: 4
    Number: 5
    Parenthesis: )
  Number: 6
  Parenthesis: )

Yet Another Config File Reader for Go (Golang)

I’ve been working up to some larger projects as I learn Go.  One of my current projects now needs the ability to parse a configuration file.

I’ve looked at a few of the existing options.  There are parsers for Windows-style INI files, YAML files, JSON files, and one for Java-like Properties files.

I’ve always preferred the simplicity of reading a configuration file into a map or associative-array construct, so I thought I’d write my own.

The project will be maintained at this Github repository:

https://github.com/jimlawless/cfg

My basic requirements were:

• A “#” character at the beginning of a line denotes a comment.
• A blank line should be ignored.
• All key/value pairs should start with a non-blank identifier name at the beginning of a line.
• An equals-sign ( “=” ) should separate the key from the value with no whitespace on either the left or right side.
• While the configuration file reader will read files that look like Java Properties files, escape-sequences in the value portion of the line will not be evaluated.

The file test.cfg will be used as a sample config file.  Its contents are as follows:

# this is a comment

# blank line above
fred=flintstone of The Flintstones
barney=barney rubble
#
biff=
Onesie=Twosie

Note that the file must end with a newline or an error condition will occur.

The config file reader library I’ve created is simply called “cfg”. The sample program to use the library to load a config file into a map is as follows:

// Copyright 2013 - by Jim Lawless
// License: MIT / X11
// See: http://www.mailsend-online.com/license2013.php

package main

import (
	"cfg"
	"fmt"
	"log"
)

func main() {
	mymap := make(map[string]string)
	err := cfg.Load("test.cfg", mymap)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("%v\n", mymap)
}

The output of the above fmt.Printf() call is:

map[fred:flintstone of The Flintstones barney:barney rubble biff: onesie:Twosie]

Note that blank values are supported on the right side of the equals sign.

The cfg library exposes one function: Load(). Load() accepts a filename and a map. The file represented by the filename is loaded into a string using the following code:

func Load(filename string, dest map[string]string) error {
	fi, err := os.Stat(filename)
	if err != nil {
		return err
	}
	f, err := os.Open(filename)
	if err != nil {
		return err
	}
	buff := make([]byte, fi.Size())
	f.Read(buff)
	f.Close()
	str := string(buff)
...

I use a regular-expression to then parse the string into its significant tokens instead of dealing with the file on a line-by-line basis.

The regular-expression is :

var re *regexp.Regexp
var pat = "[#].*\\n|\\s+\\n|\\S+[=]|.*\n"

func init() {
	re, _ = regexp.Compile(pat)
}

The regular-expression first checks for a comment-line. Then it checks for a blank line. Then, it looks for a string of non-blank characters followed by the equals-sign. Then it looks for any other characters up to and including the newline character.

A check is made before any regex parsing is performed to ensure that the data ends with a newline.  If not, the code adds a newline.  ( This check was added by a contributor whose name I’ve since lost when shuffling repos.  Sorry! )

	if !strings.HasSuffix(str, "\n") {
            str += "\n";
}

The logic in the portion of the code that parses the config data discards comment-lines and blank-lines:

		if strings.HasPrefix(s2[i], "#") {
			i++
...
		} else if strings.Index(" \t\r\n", s2[i][0:1]) > -1 {
			i++

If the string token ends with an equals sign, we have a “key”. We remove the “=” and then parse for the remainder of the line removing the newline ( and a carriage-return, if present, for files edited on Windows systems ).

Please note that I force all keys to lower-case with string.ToLower() to avoid any potential issues with a user mis-keying a key based on case. The value portion of the line is kept intact.

		} else if strings.HasSuffix(s2[i], "=") {
			key := strings.ToLower(s2[i])[0 : len(s2[i])-1]
			i++
			if strings.HasSuffix(s2[i], "\n") {
				val := s2[i][0 : len(s2[i])-1]
				if strings.HasSuffix(val, "\r") {
					val = val[0 : len(val)-1]
				}
				i++
				dest[key] = val
			}

…and if we encounter a token that doesn’t have an equals-sign or doesn’t conform to the simple grammar that we’re parsing, we return an error:

			return errors.New("Unable to process line in cfg file containing " + s2[i])

The initial cfg library code is as follows:

// cfg - Yet another config file reader
// License: MIT / X11
// Copyright (c) 2013 by James K. Lawless
// jimbo@radiks.net http://www.radiks.net/~jimbo
// http://www.mailsend-online.com
//
// Permission is hereby granted, free of charge, to any person
// obtaining a copy of this software and associated documentation
// files (the "Software"), to deal in the Software without
// restriction, including without limitation the rights to use,
// copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following
// conditions:
//
// The above copyright notice and this permission notice shall be
// included in all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
// EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
// OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
// HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
// WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
// FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.

package cfg

import (
	"errors"
	"os"
	"regexp"
	"strings"
)

var re *regexp.Regexp
var pat = "[#].*\\n|\\s+\\n|\\S+[=]|.*\n"

func init() {
	re, _ = regexp.Compile(pat)
}

// Load adds or updates entries in an existing map with string keys
// and string values using a configuration file.
//
// The filename paramter indicates the configuration file to load ...
// the dest parameter is the map that will be updated.
//
// The configuration file entries should be constructed in key=value
// syntax.  A # symbol at the beginning of a line indicates a comment.
// Blank lines are ignored.
func Load(filename string, dest map[string]string) error {
	fi, err := os.Stat(filename)
	if err != nil {
		return err
	}
	f, err := os.Open(filename)
	if err != nil {
		return err
	}
	buff := make([]byte, fi.Size())
	f.Read(buff)
	f.Close()
	str := string(buff)
	if !strings.HasSuffix(str, "\n") {
		return errors.New("Config file does not end with a newline character.")
	}
	s2 := re.FindAllString(str, -1)

	for i := 0; i < len(s2); { 		if strings.HasPrefix(s2[i], "#") { 			i++ 		} else if strings.HasSuffix(s2[i], "=") { 			key := strings.ToLower(s2[i])[0 : len(s2[i])-1] 			i++ 			if strings.HasSuffix(s2[i], "\n") { 				val := s2[i][0 : len(s2[i])-1] 				if strings.HasSuffix(val, "\r") { 					val = val[0 : len(val)-1] 				} 				i++ 				dest[key] = val 			} 		} else if strings.Index(" \t\r\n", s2[i][0:1]) > -1 {
			i++
		} else {
			return errors.New("Unable to process line in cfg file containing " + s2[i])
		}
	}
	return nil
}

I intend to use this library in an upcoming project.

Filling a Slice Using Command-line Flags in Go (Golang)

I wanted to be able to specify a particular command-line flag more than once in a Go program. I was about to throw my hands up in despair because I didn’t think that the Go flag package could process multiple instances of a command-line flag. I was wrong.

While I was tempted to write my own command-line options parser, I chose to find the idiomatic way to approach the problem.  If I have learned nothing else from my GoMentors, I have learned to try to follow the idioms and to try not to reinvent the wheel.

I found an example by visiting http://golang.org/pkg/flag/ . I had to search for the string “FlagSet” in my browser.  Immediately under the paragraph where the word “FlagSet” first appears, is a clickable item labeled “Example”.  Click the example item and take a look at the code.

I copied the code and toyed with it until I thought I understood it.  Then, I tried to simplify it and rewrite it.  My example program will simply accept one or more command-line flags with the label -i.  Each argument to -i should be an integer.  I want to be able to specify -i multiple times on the command-line.  The program should populate a slice of integers while adhering to the above command-line syntax.

Here’s my code … flagstuff.go :

// Copyright 2013 - by Jim Lawless
// License: MIT / X11
// See: http://www.mailsend-online.com/license2013.php
//
// Bear with me ... I'm a Go noob.

package main

import (
	"flag"
	"fmt"
	"strconv"
)

// Define a type named "intslice" as a slice of ints
type intslice []int

// Now, for our new type, implement the two methods of
// the flag.Value interface...
// The first method is String() string
func (i *intslice) String() string {
	return fmt.Sprintf("%d", *i)
}

// The second method is Set(value string) error
func (i *intslice) Set(value string) error {
	fmt.Printf("%s\n", value)
	tmp, err := strconv.Atoi(value)
	if err != nil {
		*i = append(*i, -1)
	} else {
		*i = append(*i, tmp)
	}
	return nil
}

var myints intslice

func main() {
	flag.Var(&myints, "i", "List of integers")
	flag.Parse()
	if flag.NFlag() == 0 {
		flag.PrintDefaults()
	} else {
		fmt.Println("Here are the values in 'myints'")
		for i := 0; i < len(myints); i++ {
			fmt.Printf("%d\n", myints[i])
		}
	}
}

Let’s dissect the code one section at a time … not necessarily in the order presented in the source code above.

First, I define a type called intslice that refers to a slice of ints:

type intslice []int

Later, I define a variable named myints of type intslice.

var myints intslice

Later in the code, I’m going to be calling flag.Var() passing in &myints as the first argument.  The type of the first value to flag.Var() must conform to the flag.Value interface which is defined as:

type Value interface {
    String() string
    Set(string) error
}

I must now define a String() method and a Set() method for my intslice type:

func (i *intslice) String() string {
	return fmt.Sprintf("%d", *i)
}

func (i *intslice) Set(value string) error {
	fmt.Printf("%s\n", value)
	tmp, err := strconv.Atoi(value)
	if err != nil {
		*i = append(*i, -1)
	} else {
		*i = append(*i, tmp)
	}
	return nil
}

The above methods will be called by the parsing engine in the flag package when I invoke flag.Parse(). In the String() method, I need to return a string-representation of the argument. In the Set() method, I then need to append the string value to the specified intslice variable by first converting value to an int variable named tmp. If an error occurs during conversion, I append an int value of -1 to the intslice variable.

The main body looks like this:

func main() {
	flag.Var(&myints, "i", "List of integers")
	flag.Parse()
	if flag.NFlag() == 0 {
		flag.PrintDefaults()
	} else {
		fmt.Println("Here are the values in 'myints'")
		for i := 0; i < len(myints); i++ {
			fmt.Printf("%d\n", myints[i])
		}
	}
}

Here are a few sample command-line invocations and the output that they produce ( I’ve added a blank line between each command and the counterpart response lines for clarity):

No parameters. I’ve added a check to make sure that more than zero flags are specified.

flagstuff

  -i=[]: List of integers

Let’s specify an invalid flag (-x):

flagstuff -x

flag provided but not defined: -x
Usage of flagstuff:
  -i=[]: List of integers

Let’s specify -i without an argument:

flagstuff -i

Usage of flagstuff:
  -i=[]: List of integers

Now, let’s specify a single -i parameter with an integer value:

flagstuff -i 5

5
Here are the values in 'myints'
5

At each invocation of the intslice.Set() method, I display the string that has been passed in so that I could observe the mechanics of the parsing process. In each example that provide arguments for -i, we’ll first see those values, then we’ll see what the slice contains via the for loop that occurs just a little later in the code.

Let’s specify a string instead of an int as an argument:

flagstuff -i twelve

twelve
Here are the values in 'myints'
-1

Note that this causes the error condition in the call to strconv.Atoi(). I have chosen to add the value -1 to the slice when the argument doesn’t cleanly parse as an integer. You may choose to handle the error differently.

Here is an example with three valid integers:

flagstuff -i 5 -i 6 -i 7

5
6
7
Here are the values in 'myints'
5
6
7

Note that the example at golang.org contains a section that splits the string value passed to Set() based on the presence of the comma character. This allows that code to also accept multiple arguments to a single command-line flag. I have chosen to avoid doing that to simplify my example.

Knowing how to handle multiple occurrences of a given flag without customizing the command-line parser is going to be very helpful for a couple of programs that I plan to write. I’m glad that I spent the time going over the example golang.org code. I hope to tinker with more exotic command-line processing features of the flag package in the near future.