All of the RDMS systems I’ve worked with have built in System Views ( or tables ) that contain data about the structure and contents of your database.  These views are there to help you with non data tasks.  For the sake of this post we’re only going mention the INFORMATION_SCHEMA, which is the system view I use most in SQL Server, and MySQL.  For your reference, here are links to INFORMATION_SCHEMA docs for four popular RDMS systems.

1. SQL Server – http://msdn.microsoft.com/en-us/library/ms186778.aspx
2. MySQL – http://dev.mysql.com/doc/refman/5.5/en/information-schema.html
3. Oracle – http://download.oracle.com/docs/cd/E17952_01/refman-5.0-en/information-schema.html
4. DB2 – http://www.tar.hu/sqlbible/sqlbible0100.html

If you work with SQL Server, you’ll be happy to know you can see all of the System Views in Management Studio by default.

System Views in the AdventureWorks database

If you work with MySQL, you can see the INFORMATION_SCHEMA database in MySQL Workbench, but you’ll have to enable it in Preferences first.

Edit > Preferences > SQL Editor > Check 'Show Metadata Schemata'

Now for the MySQL users, turning on the Metadata Schemata is not required in order to query those system tables.  Assuming your user has appropriate permissions, you can always fire up Workbench and fire a query like this.

SELECT *
FROM INFORMATION_SCHEMA.Columns
WHERE Column_Name LIKE '%user%'

MySQL has INFORMATION_SCHEMA tables too!

Now that you know about INFORMATION_SCHEMA, you should be happy to know this is part of the SQL-92 standard. Which means whatever sql scripts you write in MySQL, will most likely work with SQL Server as well. So go ahead and start querying your database, you’ll probably find uses for the INFORMATION_SCHEMA in your daily life really quickly.

I’m going to assume you’ve got the general idea here, so here are a few sql scripts that I’ve used over the years.  The first three are informational queries, and the last three are stored procedures that generate .NET or Coldfusion code based off of a table name.

Find all tables with the column EmployeeID

SELECT	*
FROM	INFORMATION_SCHEMA.COLUMNS
WHERE	COLUMN_NAME LIKE 'EmployeeID'

Which tables have an EmployeeID column?

How many tables have the word employee in the name

SELECT	*
FROM	INFORMATION_SCHEMA.TABLES
WHERE	LOWER(TABLE_NAME) LIKE '%employee%'

How many %employee% tables are in the DB?

How many SPROCs and UDFs does our database contain?

SELECT	s.SPROCs, f.UDFs
FROM
(
	SELECT	COUNT(ROUTINE_NAME) AS 'SPROCs', NULL AS 'UDFs'
	FROM	INFORMATION_SCHEMA.ROUTINES
	WHERE	ROUTINE_TYPE = 'PROCEDURE'
) AS s,
(
	SELECT	NULL AS 'SPROCs', COUNT(ROUTINE_NAME) AS 'UDFs'
	FROM	INFORMATION_SCHEMA.ROUTINES
	WHERE	ROUTINE_TYPE = 'FUNCTION'
) AS f

How many stored procedures and user defined functions are in the DB?

The stored procedures listed below all accept an incoming table name, then reads data from INFORMATION_SCHEMA to generate code for the specified database table.  Instead of listing the raw SQL, I’m just showing a sample result and allowing you to download the raw SQL sprocs.

Generate C# ADO.NET to VO Stored Procedure

Build custom C# ADO.NET to custom VO fillerup code

download stored procedure

Generate .NET VO class Stored Procedure

download stored procedure

Generate Coldfusion CFFunction with CFQUERY Stored Procedure

Generate Coldfusion CFFunction with CFQUERY for specified table name

download stored procedure

I use these System Views almost daily, and there are a lot of scenarios where my job would take a lot longer to do if I didn’t know about this information.  The INFORMATION_SCHEMA also feeds my passion to build code that makes code, just love it.  Hopefully this nickel tour was enough information to help somebody out.

There is no rhyme or reason here, just some things I felt need to be repeated.

1. PreserveSingleQuotes()

This one came in really handy on a project requiring large text files to be imported into a MySQL database.  I used Coldfusion to upload and read the files into large INSERT chunks using MySQL’s multi – row INSERT syntax. Code built the VALUES portion of the SQL, then I just fed the data into a function for insertion.

INSERT INTO tbl_name (a,b,c) VALUES(1,2,3),(4,5,6),(7,8,9);

	<cfquery name="insert_data" result="insert_result" DATASOURCE="#request.dsn#" USERNAME="#request.dbuser#" PASSWORD="#request.dbpswd#">
	INSERT INTO table
	( column1, column2, column3, column4, column5, column6, column7 )
	VALUES
	#PreserveSingleQuotes( insert_values )#
	</cfquery>

2. What if GENERATED_KEY doesn’t work?

If you’re using MySQL and the GENERATED_KEY property of your cfquery objects isn’t populating, you can use LAST_INSERT_ID instead.

SELECT LAST_INSERT_ID();

3. Use ArrayAppend when building strings

Classic performance tuning tip for just about any programming language.  Here I’ll give a Coldfusion example and keep it dead simple.  If you ever have to concatenate strings in code, user ArrayAppend instead.  Here are two loops that do the same thing.  If you run this code, you should notice loop1 takes forever, and loop2 is smoking fast.

slow…..

<cfscript>
xx				= 100000;
insertString	= "";

// do the loop
while( xx > 0 ) {
	insertString &= xx & " ";
	xx--;
}

WriteOutput( insertString);
</cfscript>

FAST!

<cfscript>
xx				= 1000000;
insertArray		= ArrayNew(1);
// do the loop
while( xx > 0 ) {
	ArrayAppend( insertArray, xx & " " );
	xx--;
}

WriteOutput( ArrayToList( insertArray, " " ) );
ArrayClear( insertArray );
</cfscript>

4. If CSV, then CHR

This one is simple, if you find yourself creating CSV or any other text file, use special characters when dealing with single and double quotes, etc.

• chr(9) = Tab
• chr(34) = ” double quote
• chr(39) = ‘ single quote

And if you’re not sure of the correct code for the character you’re looking to use, just wrap that character in ASC() and WriteOutput to the page.

5. Use CFMail with GMail

This is a no brainer, but with how difficult sending email can be with other languages, I’m mentioning it here.

Application.cfc

<cfscript>
	APPLICATION.mail.server		= "smtp.gmail.com";
	APPLICATION.mail.port		= "465";
	APPLICATION.mail.ssl		= true;
	APPLICATION.mail.user		= "gmailAccount";
	APPLICATION.mail.pswd		= "gmailPasssword";
</cfscript>

Emailer.cfm

<cfmail to="work@ericfickes.com"
		bcc=""
		from="web@master.com"
		subject="sending mail is easy with Coldfusion"
		server="#application.mail.server#"
		useSSL="#application.mail.ssl#"
		port="#application.mail.port#"
		username="#application.mail.user#"
		password="#application.mail.pswd#">
#emailBody#
</cfmail>

6. CFSCRIPT doesn’t know NULL?

Another tip from the land of importing and exporting data.  While working with query objects in CFScript, for some reason I could never accurately detect for NULL values.  I tried all sorts of detection schemes and ended up just writing a hacky fail safe.  Please, if you have a better suggestion for *easy* NULL detection in CFSCript, add it in the comments below.

utils.cfc

<!---  Simple value getter with try / catch to get around NULL values
This function originated in a script where we always needed a " " even if
the value from the database was null.
--->
<cffunction name="qryGetString" access="public" returntype="string">

<cfargument name="data" type="string">

<cftry>
	<cfscript>
	return #data# & " ";
	</cfscript>

	<cfcatch type="Any">
		<cfreturn " "/>
	</cfcatch>
</cftry>

</cffunction>

Exporter.cfm

// largeish loop
tab = chr(9);
for( xx = 1; xx <= queryObj.RecordCount; xx++ )
{
	// start the row
	this_row = 	utils_cfc.qryGetString( queryObj.FirstName[xx] ) & tab &
			utils_cfc.qryGetString( queryObj.MiddleName[xx] ) & tab &
			utils_cfc.qryGetString( queryObj.LastName[xx] ) & tab &
			utils_cfc.qryGetString( queryObj.Suffix[xx] ) & tab &
			utils_cfc.qryGetString( queryObj.MedicalTitle[xx] ) & tab &
			utils_cfc.qryGetString( queryObj.email[xx] ) & tab &
			utils_cfc.qryGetString( queryObj.practice_phone[xx] ) & tab &
			utils_cfc.qryGetString( queryObj.practice_fax[xx] ) & tab &
			utils_cfc.qryGetString( queryObj.practice_name[xx] ) & tab &
			utils_cfc.qryGetString( queryObj.practice_address1[xx] ) & tab &
			utils_cfc.qryGetString( queryObj.practice_address2[xx] ) & tab &
			utils_cfc.qryGetString( queryObj.practice_city[xx] ) & tab &
			utils_cfc.qryGetString( queryObj.practice_state[xx] ) & tab &
			utils_cfc.qryGetString( queryObj.practice_zipcode[xx] ) & tab &
			utils_cfc.qryGetString( queryObj.hospital_affiliation[xx] ) & tab;

	// do stuff with the data
}

7. How I find list items

Ever notice the different behavior in the Coldfusion ListFind commands?  I ended up writing my own ListHasValue function in order to find exact pattern matching in a list.  I had a list of role ids in a list, and just couldn’t get the built in functions to tell me when my id was in the list without also matching on other ids.  This one makes sense when you run some code.

The top of this sample as my custom ListHasValue() command, and the lower half does three simple loops counting from 1 to 100, and using ListFind, ListContains, and ListHasValue for number checking against the same list.

<!---
	Use this to do an exact pattern check for a value in a list.
	This is useful inside of a loop checking numbers against a number list.

	EX :

	list = "91, 92"

	if you loop from 1 - 100, ListFind and related CF functions will match
	on 1, 2, and 9.  Not just 91 and 92

	Use this when you're looping and you ONLY want to match on 91, or 92

--->
<cffunction name="ListHasValue" access="public" returntype="boolean">

	<cfargument name="list" required="yes" type="string">
	<cfargument name="value" required="yes" type="any">

	<cfscript>
		// clean up to be safe
		list = trim( toString( list ) );

		// check to see if we have a *possible* match
		position = ListContains( list, value ) ;

		if( position > 0 )
		{
			// NOTE : KEEP THE TRIM AND TOSTRING
			found_value = trim( toString( ListGetAt( list, position ) ) );

			if( Compare( value, found_value ) == 0 )
			{
				return true;
			}
		}

		// no match for you!
		return false;
	</cfscript>

</cffunction>

<cfscript>
list	= "1, 11, 21, 31, 41, 51, 61, 71, 81, 91, 99";
xx		= 1;

WriteOutput( "List = " & list & "<hr>");

while( xx < 100 ) {
	if( ListContains( list, xx ) > 0 )
	{
		WriteOutput( "ListContains found " & xx & "<br>" );
	}
    xx++;
}

WriteOutput("<hr />");

xx = 1;
while( xx < 100 ) {
	// ListFind
	if( ListFind( list, xx ) > 0 )
	{
		WriteOutput( "ListFind found " & xx & "<br>" );
	}
    xx++;
}

WriteOutput("<hr />");

xx = 1;
while( xx < 100 ) {
    // Eric's ListHasValue
	if( ListHasValue( list, xx ) )
	{
		WriteOutput( "ListHasValue found " & xx & "<br>" );
	}
	xx++;
}
</cfscript>

If you run this code on your Coldfusion server, you should notice the following results.  ListContains matches single digits from the loop that do not really exist in the list.  ListFind only finds the number 1?  And finally, my function does exactly what I needed it to do.  Tell me when a specific number exists in a list.

Coldfusion ListFind functions don't always behave how I want them to

8. Make PDFs faster

This could easily be it’s own topic, but I’ll say one thing about making PDFs faster with CFDocument.  Only put final content between <cfdocument> and </cfdocument>.  That is, if you have any processing code, cfqueries, cfloops, inside of your cfdocument tag, your cfml page is running slower than it needs to be.  Here’s a simple example of one of my cfml pages that has only final content in the cfdocument tags.

The key to this example is moving all of my content creation code into an external file, then including at the top of my page. I always do a check for my main PDF_BODY variable, and then spit out my PDF document.

<cfinclude template="code/export_pdf_codefile.cfm">

<cfif PDF_BODY NEQ "">

    <cfdocument	name="provider_profile"
                format="PDF"
                pagetype="A4"
                mimetype="application/pdf"
                orientation="portrait"
                margintop="0"
                marginbottom="0.2"
                marginleft="0.2"
                marginright="0.2"
                >
        <cfoutput>#PDF_BODY#</cfoutput>
    </cfdocument>

    <!--- send directly to client --->
    <cfheader name="Content-Disposition" value="attachment; filename=#filename#">
    <cfcontent type="application/pdf" variable="#provider_profile#">

<cfelse>
	No PDF content found
</cfif>

9. Use parameterized queries

This is a tip for all server side programmers whether you use Coldfusion, ASP, JSP, PBJ.  Use parameterized queries when doing any database interaction.  It’s too easy not to use, and you get protection from SQL Injection, as well as enforcing proper data types when speaking to your database.  This is something all server programmers should do regardless of your language, the sample below is for Coldfusion.

BAD

<cfquery name="tblInsert" datasource="myDb">
INSERT INTO	myTable
( col1, col2, col3, col4 )
VALUES
( '#Form.field1#', '#Form.field2#', '#Form.field3#' )
</cfquery>

GOOD

<cfset val1 = Form.field1>
<cfset val2 = Form.field1>
<cfset val3 = Form.field1>

<cfquery name="tblInsert" datasource="myDb">
INSERT INTO	myTable
( col1, col2, col3, col4 )
VALUES
(
    <cfqueryparam cfsqltype="cf_sql_varchar" value="#val1#" />,
    <cfqueryparam cfsqltype="cf_sql_varchar" value="#val2#" />,
    <cfqueryparam cfsqltype="cf_sql_varchar" value="#val3#" />
)
</cfquery>

10. Where’d the time go?

For the good programmers already using parameterized queries, ever insert a timestamp into your database and find out the date is correct, but the time is always 12:00:00?  Take a closer look at the cfsqltype in your cfqueryparam, I had this exact problem and here’s what happened.

Using cf_sql_date does not include the full date and timestamp, just the date.

<cfqueryparam cfsqltype="cf_sql_date" value="#paymentDate#" />

Using cf_sql_timestamp includes the full date and timestamp I was looking for.

<cfqueryparam cfsqltype="cf_sql_timestamp" value="#paymentDate#" />

11. Stored Procedures are a little different

This last one isn’t much of a tip, but more of a reminder to myself.  I do so much database work that stored procedures are just queries to me, but not so to Coldfusion and the CFQuery tag.  If you want to get data from a stored procedure, you need to use the CFStoredproc tag.  Here’s a sample of passing one argument into a stored procedure, and how to get the resulting data.

<cfstoredproc	datasource="myDb" procedure="GetDataFaster">

	<cfprocparam type="in" cfsqltype="cf_sql_integer" value="#inputVar#" />

	<!--- specify sproc result here, cfstoredproc res != returned recordset --->
	<cfprocresult name = sprocResult>

</cfstoredproc>

<cfreturn #sprocResult.ColumnFromQuery#>

After finishing this post I ran into all sorts of other strange issues and restarted using a Web Appliction instead of a plain old Website.  Between IntelliSense not showing any classes, to project reference issues, I couldn’t figured it out in time.  I’m sure there’s a way, I just had to move on.  So maybe this ramble below will be helpful for someone.

Just ran into something quirky with Visual Studio 2008′s new ADO.NET Entity Data Model wizard.  While working on an ASP.NET 3.5 website ( not a codebehind web application ) I was trying to get the ADO.NET Entity Data model wizard to work with MySQL and ran into a probable Visual Studio bug.  To sum up the issue, if you are going to add a new edmx to your project, do NOT save it to the App_Code folder initially.  Put it in your root folder, compile your project, then move the edmx where you’d like.

Assuming you’ve already created your ASP.NET Website project, here’s how you reproduce this issue.

Right click your project and left click ‘Add New Item’

Add New Item

Select ADO.NET Entity Data Model, name it, select your language of preference

ADO.NET Entity Data Model

Click Yes to the ‘Store in App_Code’ prompt

Place file in 'App_Code' folder

Complete the new Entity Data Model wizard

See this tutorial if you have not done this before

Compile project after completing wizard

Invalid token 'void' in class, struct, or interface member declaration

At this point your project should have a new.edmx file located inside of the App_Code folder, but the project won’t build without failing.  If you are stuck in this predicament, follow this workaround.

Move .edmx to root folder and rebuild

WORKAROUND : move edmx to root folder, then recompile

After moving your edmx file to the root folder you should be able to compile without problem.  Assuming this solves your problem, you should be able to move your edmx file to the App_Code folder without problem.  Seems like an initial compile problem.

Requirements :

• MySQL 5.x installed on your local machine
• MySQL command line client on your local machine
• Access to phpMyAdmin on your source MySQL machine

Assumptions:

• You have access to the MySQL command line client.
• You have the root password for your local MySQL instance
• You are comfortable using phpMyAdmin and MySQL command line client

Instructions:

1. Log into phpMyAdmin on your source MySQL machine.
2. Click the Export link on the lower part of the home page.
3. Select your database in the ‘Export’ box on the left.
4. Make sure SQL is selected under the database box.
5. In the ‘Options’ box on the right, make sure Structure and Data are selected.
6. Make sure to check ‘Add DROP TABLE / DROP VIEW’ as well.
7. Towards the bottom, check the box next to ‘Save as file’. This will be the file we import later.
8. Click the GO button, and your browser should prompt you to download your SQL file. *Make note of where you saved your exported sql file. I saved mine to d:\downloads\import.sql because it’s an easy path to use later.
9. Open your local MySQL Command Line Client. ( From MySql Program menu or type “mysql -u root -p” in the run dialog box ).
10. Enter your root password when prompted.
11. Import your database by running this command “mysql \. d:\downloads\import.sql“. Be sure to adjust the path to match where you saved your sql file from step 8.
12. Now you should see your database listed when you run the command “show databases;“

Here are instructions for doing the same thing on OS X

1. Follow steps 1 through 8 above to created your sql export file
2. Connect to mysql “/usr/local/mysql/bin/mysql -u root -p”
3. {enter root password}
4. If it’s not already created, created your databse “create database {your db name};”
5. Select your database “user {your db name};”
6. “mysql \. /users/ericfickes/Downloads/mydbfile.sql”

Be sure to adjust the paths listed to match your MySQL install, and the location where you downloaded the sql file.