Sunday, November 6, 2011

Connecting MySQL to GNU Octave in Ubuntu

In my little world, this is, officially, A Very Big Deal (tm).  
MySQL features a rich API.  GNU Octave has a database package. 
Unfortunately I don't know API from DPI, and Octave's db package pretty much doesn't work for anybody (including me), according to my google results.


So I wrote my own.. it was:

  1. my first foray in .oct file creation, C++, and APIs
  2. successful!  thanks largely to the extensive help available all over the internet

Regardless, I'm disproportionately pleased with myself.  


Note you'll need both octave and MySQL header files, and my goal was to get a .oct that could retrieve data FROM the database, which is different than stuffing data TO the database.


The code below takes in four strings, queries MySQL, and if the query was successful, returns a cell variable to octave.

/*
 * SQL2m.cpp
 *
 *  Created on: Nov 5, 2011
 *      Author: nzvyyx
 */


#include "/usr/include/mysql/mysql.h"
#include "octave/oct.h"
#include "octave/Cell.h"
#include <string>
using namespace std;


#define NUM_INPUTS_REQUIRED (4) // 0 is file name, 1: user name, 2: password, 3: database name, 4: SQL query


#define  NoError 0
#define  Error_NoInputs (NoError + 1)
#define  Error_SQL_QueryError (Error_NoInputs + 1)
#define  Error_UnexpectedTermination (Error_SQL_QueryError + 1)
#define  Error_UserNameNotAString (Error_UnexpectedTermination + 1)
#define  Error_UserPasswordNotAString (Error_UserNameNotAString + 1)
#define  Error_DatabaseNotAString (Error_UserPasswordNotAString + 1)
#define  Error_SQL_Query_NotAString (Error_DatabaseNotAString + 1)
#define  Error_NoOutputs (Error_SQL_Query_NotAString + 1)
#define  Error_SQL_CnxnFailed (Error_NoOutputs + 1)


#define  *ErrorMsgs[] = {
    ">> SUCCESS <<",
    "not enough inputs, expected (name, password, db_name, SQL_query)",
    "SQL query error",
    "unexpected program exit",
    "non-character user name",
    "non-character user password",
    "non-character database name",
    "non-character SQL query",
    "no output assigned",
    "SQL failed to connect",
    NULL
};


char *server = "localhost";
char *username = NULL;
char *password = NULL;
char *database = NULL;
char *SQL_query = NULL;
string str;
MYSQL *conn;
MYSQL_RES *res;
MYSQL_ROW row;
unsigned long long totalrows, numfields;
int _n;
octave_idx_type _row, _col;
octave_value_list  retval;


DEFUN_DLD (SQL2m, args, nargout,
        "[SQL_result] = SQL2m(database, user_name, user_password, SQL_query)")
{
  if (nargout < 1)
  {
    retval(0) = ErrorMsgs[Error_NoOutputs];
    return retval;
  }
  if (args.length() < NUM_INPUTS_REQUIRED)
  {
    retval(0) = ErrorMsgs[Error_NoInputs];
    return retval;
  }
  for (_n = 0; _n < NUM_INPUTS_REQUIRED; _n++)
  {
    if (!(args(_n).is_string()))
    {
      retval(0) = ErrorMsgs[Error_UserNameNotAString + _n];
      return retval;
    }
  }


  str = args(0).string_value();
  username = new char[str.size() + 1];
  strcpy(username, str.c_str());
  str = args(1).string_value();
  password = new char[str.size() + 1];
  strcpy(password, str.c_str());
  str = args(2).string_value();
  database = new char[str.size() + 1];
  strcpy(database, str.c_str());
  str = args(3).string_value();
  SQL_query = new char[str.size() + 1];
  strcpy(SQL_query, str.c_str());


  conn = mysql_init(NULL);


  if (NULL == mysql_real_connect(conn, server,
      username, password, database, 0, NULL, 0))
  {
    retval(0) = ErrorMsgs[Error_SQL_CnxnFailed];
    return retval;
  }


  if (0 != mysql_query(conn, SQL_query))
  {
    retval(0) = ErrorMsgs[Error_SQL_QueryError];
    return retval;
  }


  res = mysql_store_result(conn);


  totalrows = mysql_num_rows(res);
  numfields = mysql_num_fields(res);
  Cell DATA(totalrows, numfields);


  _row = 0;
  while (row = mysql_fetch_row(res))
  {
    for (_col = 0; _col < numfields; _col++)
    {
      DATA(_row, _col) = octave_value(row[_col]);
    }
    _row++;
  }


  mysql_free_result(res);
  mysql_close(conn);


  return octave_value(DATA);
}

Here's a simple octave .m file to build the .oct file  & test for  results.
You'll need to a MySQL running, and the dummy placeholders for user name, password, etc need functional values from you.
% BuildAndTestSQL2m.m
clear all; clc; 

% build required MySQL API:
% $ sudo apt-get install libmysqlclient-dev
%
% directories given by command line:
% $ mysql_config --cflags
% and
% $ mysql_config --libs
mkoctfile  -I/usr/include/mysql  -L/usr/lib/mysql -lmysqlclient  SQL2m.cpp 

disp("change this script to specify your username, password, database name and SQL query !")
username = 'user';   
password = 'password';
dbname = 'dbname';
SQL_query = 'select field1, field2 from table';

[status, output] = system('rm SQL2m_tests.txt');
diary SQL2m_tests.txt
help SQL2m
% test with combinations of insufficient argument counts
SQL2m()
results = SQL2m()
results = SQL2m(username)
results = SQL2m(username, password)
results = SQL2m(username, password, dbname)

% this should return a cell array - success!
results = SQL2m(username, password, dbname, SQL_query)
diary off

Saturday, July 23, 2011

GNU Octave Custom Colormap

Hello, Unlikely Reader - 
So I wanted to rescale color 3D plot with a more agreeable set of values.  Octave handles this, but I thought I'd write up how because it isn't an everyday task and I'm sure I'll forget how I did it.


Here's some data plotted with the default colormap:


The plot is fine but the wide range of color was subjectively implying more curvature in the data than I meant to.  Of course that's all subjective but the data was supposed to make a point and the colormap was working against it.  






Here's a 3D view that makes my point better, but 3D views on flat surfaces are harder to interpret; for example, the diagonal "valley" of low spots gets a little lost:



So I built my own lowest to highest Red-Yellow-Green colormap and applied it; here's how.
n_levels = (10-7)/0.1 + 1;  % numbers of interest range from 7 to 10


% build RGB channels from red to yellow:
red2yellow_chRed = linspace(1, 1, n_levels/2);
red2yellow_chGreen = linspace(0, 1, n_levels/2);
red2yellow_chBlue = linspace(0, 0, n_levels/2);


% build RGB channels from yellow to green:
yellow2green_chRed = linspace(1, 0, n_levels/2);
yellow2green_chGreen = linspace(1, 1, n_levels/2);
yellow2green_chBlue = linspace(0, 0, n_levels/2);


% create the colormap by stacking and concatenating the channel vectors::
RYG_colormap = [ ...
[red2yellow_chRed'; yellow2green_chRed'] ...
[red2yellow_chGreen'; yellow2green_chGreen'] ...
[red2yellow_chBlue'; yellow2green_chBlue'] ...
];

% set the colormap
colormap(RYG_colormap);


The plot automatically updates and helps support my point that there's not so much curvature in the data.















If you don't like it, you can easily switch back:
colormap("default");
Or if black and white is better for you, there's a predefined grayscale map:
colormap(gray);
















There you go.  Colormaps.   Happy (r)Octave-ing!

- nzvyyx

Tuesday, July 19, 2011

Shuttleworth Kicks Ballmer's ASS

Thanks to LiLi's Live USB Creator I've got a nice clean A:B comparison of M$ vs Linux.  Today's heavy lifting for the OS was to run a perl script (which worked fine under XP):

My shiny new Windows 7 FAILS:




...while Ubuntu running from a 4Gb LiLi Live USB works just fine, thank you:


Best of all, thanks to LiLi's VirtualBox install, I don't even have to choose between operating systems!  Here's Ubuntu running in a virtual machine hosted by my L7 OS.


Maybe I'll be able to get my job done after all.


- nzvyyx

Saturday, July 16, 2011

VNC... high quality & free remote access

Dear Blogspot/Blogger.Com:
Thank you for your wonderful service.  I noticed a sidebar ads pushing a for-pay remote PC access program - which will remain unnamed.  It appears to be a scam to continually charge hapless users for free software.


Dear Reader:
If you need to access computers remotely, look into VNC... you'll probably notice that web searches prioritize non-free programs... caveat emptor!  Those ripoff artists have a lot of time on their hands to bury the high quality, cross-platform free stuff with their own web pages and link overload.


- nzvyyx

Friday, July 15, 2011

... and L7 once again FAILS

Yesterday I had MySQL up and running on my overpriced WindoesNT laptop as provided by MegaCorp Inc.  I logged in/out/in/out both as root and a user.  I pulled in my ginormous database that I had saved from my old burn.  I entered data, set up queries and forms in LibreOffice Base.   All appeared to be working fine.  I was a happy user.


Last night I shut down.  Restart the machine in the morning.... login denied.  No MySQL access.  Denied.  Hopefully I can find a way to get my data back, but regardless that was the last straw and it's official:


I HATE WINDOWS 7.


This is probably due to UAC resetting on a reboot, but no matter what: data loss is unacceptable... and since I can't get to the data, it counts as lost.  No matter what, erratic behavior is frustrating as hell for a user.  This "feature" of resetting UAC on reboots is a built-in frustration.


Next plan is to bring in my cheap-O linux box and do all my real work on it.




- nzvyyx

Thursday, July 14, 2011

Connecting Base to MySQL

Notes on getting my Ubuntu machine to connect Base to MySQL:
sudo apt-get install libmyodbc


Then add a couple of config files (thanks, OpenOffice Wiki)
































 Let Base know what's up:






Connect!






- nzvyyx

Wednesday, July 13, 2011

bash to dos to bash (thanks, Git!)

When I installed Git, it gave me several context menus to get the bash prompt.  On my new Win7 machine this makes reaching the bash prompt much easier & faster than finding a windows command line... but wait I can easily get a dos prompt FROM the bash prompt.  Woops, I'm never learning any more Powershell or whatever.  C Ya, Ballmer!


As an entertaining aside it appears that the "7" in Windows 7 is a minor release level.  Ya like I didn't already know it was Vista.




- nzvyyx

Tuesday, July 12, 2011

Plot Digitizer

If you need to pull data from a graphic, Plot digitizer is for you.  Fairly small memory footprint & intuitive controls, and it's cross-platform (which is a big deal for me these days...)
This might help you get it set up:



sudo mkdir PlotDigitizer
cd PlotDigitizer
wget http://downloads.sourceforge.net/project/plotdigitizer/plotdigitizer/2.5.1/PlotDigitizer_2.5.1_Linux_and_Others.tgz
tar -xvzf PlotDigitizer_2.5.1_Linux_and_Others.tgz


- nzvyyx

Thursday, July 7, 2011

Finally a Ubuntu complaint

Aha! Ubuntu doesn't support SciTE.  Bummer.
And it looks like the gtk version that shipped w/Maverick was a little stale, so I couldn't just build my own SciTE per the handy instructions in the README file.  Bummer.


And it was pretty easy to fix... Bummer!


First I got up to speed on GTK:
sudo apt-get install libgtk2.0-dev



Then, following the README included in the SciTE source download, I built Scintilla and then SciTE.  Only catch was I didn't want it installed on my desktop, so I unzipped the files into /usr/local to build & install there.

cd scite227
cd scintilla/gtk
make
cd ../..
cd scite/gtk
make
sudo make install


- nzvyyx

Sunday, July 3, 2011

L7 vs Linux

"Computer experts" are a byproduct of the Micro$oft's planned obsolescence business model. 


Micro$oft is a huge company.  They need money or they'll go out of business.  So, every few years they "sell" you another software product.  Maybe the product you were using worked fine... doesn't matter!  They need revenue and the product you already bought isn't generating any cash for them.


From one perspective they didn't really "sell" you anything anyway.  You are pretty much leasing your operating system.  You don't own your copy of Windows; it is licensed to you.  


Anyway, to assure their own perpetuity, every few years they "sell" you something.  It has to be different than their last version or people might suspect they're being scammed.  This process is called "upgrading" rather than "constant churn," which is how I think of it.  


Given that computers are difficult, picky and arcane, this constant churn creates a job position  for people who are good at memorizing lots of detail... voilĂ , you have invented the computer expert. 


This whole rant was generated by the extreme difference between my work computer Windows7 "upgrade" vs my recent Ubuntu kernel "upgrade..." unlike L7, I have a couple of post-upgrade scripts I run for Ubuntu, one for my wireless chip and this one (if I need it).
Most of my add-ons to ubuntu are handled by looping an apt-get command on an array of package names, but because Canonical doesn't support SciTE my poor little script has to work slightly harder.    


#!/bin/bash


# copy this file to your desktop, 
# and make it executable via sudo chmod +x 
clear
sudo su


declare programs=("chromium-browser" "octave" "gnuplot" "texmaker" "mysql-server" "r-base" "gimp" "gpaint" "scite")


echo "installing ${#programs[*]} programs: ${programs[@]}"


for program in ${programs[@]}
do
  problem=$(dpkg -s $program | grep installed)
  echo Checking $program: $problem
  if [ "" == "$problem" ]; 
  then
    echo "No $program. Setting up $program"
    sudo apt-get install $program 
  else
    echo "$program installed."
  fi
  clear
done


# add headphones
sudo echo "options snd-hda-intel model=thinkpad" >> /etc/modprobe.d/alsa-base.conf


# add scite as default text editor & make visible line numbers
sudo perl -pi.bak -e 's/gedit/scite/g' /usr/share/applications/defaults.list
sudo perl -pi.bak -e 's/\Q#line.margin.visible\E/\Qline.margin.visible\E/g' 



# ubuntu doesn't support scite... this installs scite v2.27
wget http://prdownloads.sourceforge.net/scintilla/scite227.tgz
sudo cp scite227.tgz /usr/local
cd /usr/local
sudo tar -zxvf scite227.tgz
cd scintilla/gtk
sudo make
cd ../..
cd scite/gtk
sudo make
sudo make install




- nzvyyx

Thursday, June 16, 2011

.gitconfig for Windows 7

Living the dream as a "user acceptance team" pilot on a Windows 7 burn.  Why do the I.T. guys call it a "burn?"  Try Windows 7, or as I would prefer to call it, L7... you will agree that "burn" is a good descriptor.  Erratic hardware and settings, plus a BLACK SCREEN OF DEATH.


On the plus side the decision to use L7 was already made, so there is no fear or uncertainty about it, regardless of actual performance or need.  If nothing else, L7 really makes me appreciate Ubuntu.


Finally got Git working again and I thought I should post my .gitconfig or at least as much as I think I can get away with, without risking my so-called "job."



[user]
name = Dirk
email = my.email@BigOlCompany.com
[core]
editor = c:/Data/_work/Notepad++/notepad++.exe
#  whitespace=fix,-indent-with-non-tab,trailing-space,cr-at-eol
autocrlf = true
[apply]
whitespace = nowarn
[color]  
  ui = auto
[color "branch"]
current = white reverse
  local = cyan
  remote = green
[color "diff"]
  meta = yellow bold
  frag = magenta bold
  old = red bold
  new = green bold
  whitespace = red reverse
[color "status"]
  added = yellow
  changed = green
  untracked = cyan
[merge]
tool = diffmerge
[mergetool "diffmerge"]
cmd = c:/Program\\ Files/SourceGear/DiffMerge/DiffMerge.exe  --merge --result="$MERGED" "$LOCAL" "$BASE" "$REMOTE"
keepBackup = false
trustExitCode = true

Sunday, May 1, 2011

My blog has won the "Works on MY machine" Award

By posting this entry successfully, I have awarded myself this coveted prize.  

Yay for self-scoring!



Sunday, March 13, 2011

Notes from the North

International Falls, MN is called "The Icebox of the Nation" for several good reasons.  

  1. 1. it's a damned cold place
  2. the government weather station officially proves it's damned cold
  3. the city owns the trademark "Icebox of the Nation"

The weather station is right next to the airport:

             48°34'N / 93°24'W.. INL

KEDA runs a big ol' 'fridge for those of us who just can't get cold enough.  Ask for Paul, tell him Dixie sent you.  Entertainment includes shivering, warm showers, alcohol, and good food.  Get the broiled walleye at The Spot on 53, coffee at The Coffee Landing, and pretty much anything at Margarita's (Texas Fajitas were my favorite).

And tell the guys at The Viking Bar that the Kensington Stone is a hoax.

- nzvyyx

Sunday, February 6, 2011

The Tail Without The Rooster

Automotive transmissions need  friction.  Friction is a major player in getting you from one gear to another, because  friction is what forces different spinning parts to have the same  RPM.  And, because a career in engineering looks a lot like Sisyphus' career in landscaping, we engineers tend to complain.  A lot


Transmission guys love to complain about friction coefficient, and with good reason.  If you get the wrong kind of friction, those gear changes that you don't think that much about - they can turn into obnoxious events in your car.  Traditionally, we complain if the static friction coefficient is a lot higher than the dynamic coefficient.  It even gets its own name: "roostertail,"  because of the shape of the coefficient graph produced by an SAE-2 test.  


So, friction is good but "roostertail" friction is bad.  Here's a plot of simulated vehicle acceleration I created, and it's showing something that looks a lot like roostertail.  It's the little bump at the end of the shift, inside the red oval:



That looks like a roostertail friction effect.  The shift is doing its thing and then right at the end there's a bump.  Roostertail, my old frictional frenemy.

The only problem with this diagnosis is that my simulation doesn't include roostertail friction.  I know this for sure because I typed every line of it.  So where's the bump coming from?

Turns out there's plenty of ways to get a bump.  One of them is to get your shift and your converter K factor to fight it out..  Here's engine and turbine RPM, doing their gear shift thing.  Looks pretty good from here...


...but converter torque is NOT cooperating.  In fact it looks like my lookup tables flatlined, and if I had a more complete data set I betcha the bump would be worse.


So there you go, The Tail Without The Rooster.  And once again, thank you Octave.  


Also occurs to me those runs are probably a breakpoint in my simulation development, since I'm  porting my models into compiled C lately.  I'll still be using Gnuplot for visualization... but wow! looking back it's amazing how much work you can get done with Notepad++ & Octave.  On to Eclipse & MinGW now... 


- nzvyyx