Performance test - comma delimited string

NadAngel · 10-15-2012, 02:30 PM

Here is an interesting benchmark test I have made to test the performance of two methods for building a comma delimited string.

An active demo is here:
http://benjaminpajk.host22.com/test1.html

Code:

<!doctype html>
<!--Author: Benjamin Pajk-->
<html lang="sl">
	<head>
		<meta charset="UTF-8">
		<title>HTML5 Template</title>
		<meta name="description" content="">

		<meta http-equiv="X-UA-Compatible" value="IE=9"> 
		
		<!--
		<link rel="stylesheet" href="css/style.css">
		-->
		
		<script src="http://code.jquery.com/jquery-latest.js"></script>
		
		<script type="text/javascript">
		$(document).ready(function(){
				
			var numTests=50; //number of benchmark test samples
			var numIterations=1048576; //one mega (10*1024*1024=1048576) iterations
		
			var testString='';
			var start;
			var end;
			var i;
		
			var testArray=new Array();
		
			function method(){
				this.method1=-1;
				this.method2=-1;
			}
		
			/*perform benchmark test for bulding comma delimited string*/
			//method 1 checks if we need to insert the comma on every iteration
			function method1(){
				testString='';
				start =  new Date();
				insertComma=false;
				for(i=0; i<numIterations; i++){
					if (insertComma){
						testString+=',';
					}else{
						insertComma=true;
					}
					testString+='aa bb';
				}
				end = new Date();
				$('#testResult1').val(end-start);
				return end-start;
			}
			
			//method 2 inserts the comma on every iteration and removes the last comma when it finishes
			function method2(){
				testString='';
				start =  new Date();
				for(i=0; i<numIterations; i++){
					testString+='aa bb,';
				}
				testString=testString.slice(0,testString.length-1);
				end = new Date();
				$('#testResult2').val(end-start);
				return end-start;
			}
			
			//test function (recursive)
			function cmp(x){
				if (x>0){
					//recursive call with 500 ms delay
					setTimeout(function(){cmp(x-1)}, 500);
					var obj=new method();
					obj.method1=method1();
					obj.method2=method2();
					testArray.push(obj);
					$('#parallelTest').text('Testing: '+Math.round((1-x/numTests)*100)+' %');
				}else{
					//calculate the average
					var sumArr1=0;
					var sumArr2=0;
					for(var k=0; k<testArray.length; k++){
						sumArr1+=testArray[k].method1;
						sumArr2+=testArray[k].method2;
					}
					//display average results
					$('#avgResult1').val(sumArr1/testArray.length);
					$('#avgResult2').val(sumArr2/testArray.length);
					$('#testResult1').val('');
					$('#testResult2').val('');
					$('#parallelTest').text('Start parallel test');
				}
			}
			
			$('#parallelTest').click(function(){
				//reset the variables
				testArray=[];
				$('#avgResult1').val('');
				$('#avgResult2').val('');
				//call the benchmark test function
				cmp(numTests);
			});

		});

		</script> 
		
	</head>
	<body>
		<p>
		This script tests the performace of two methods for building comma delimited strings. Each test iterates 1048576 (1024*1024) times and adds "aa bb" with a comma 
		delimeter on every iteration. The test is performed 50 times for both methods and the average time per test is calculated for both methods. The results are expressed 
		in milliseconds.<br>
		</p>
		<p>
		Example: aa bb,aa bb,aa bb,aa bb,aa bb,aa bb<br>
		</p>
		<p>
		<b>method 1:</b> checks if we need to insert the comma on every iteration<br>
		<b>method 2:</b> inserts the comma on every iteration and removes the last comma when it finishes<br>
		</p>
		<p>
		Method 1:<input type="text" id="testResult1" readonly="readonly" value="">ms<br>
		Method 2:<input type="text" id="testResult2" readonly="readonly" value="">ms<br>
		Method 1 (average):<input type="text" id="avgResult1" readonly="readonly" value="">ms<br>
		Method 2 (average):<input type="text" id="avgResult2" readonly="readonly" value="">ms<br>
		<button id="parallelTest">Start parallel test</button>
		</p>
		
		<p>
		Preliminary results show, that IE9 is in favor of the first method while Chrome Firefox and Opera are in favour of the second method.<br>
		Tested browsers:<br>
		Internet Explorer 9.0.7<br>
		Firefox 16.0<br>
		Chrome (ver. 22.0.1229.94 m)<br>
		Opera 12.02<br>
		</p>
		The main difference in browser performace is in the way the function <b>testString=testString.slice(0,testString.length-1);</b> is executed.
		A good interpreter (Firefox, Opera, Chrome) will not actualy copy the string but will just delete the last cell of the existing string.
		<p>
		
		</p>
	</body>
</html>

rnd me · 10-15-2012, 04:49 PM

js performance is more about coding than browsers.

while you may have found a subtle distinction in between browsers for two ways of doing one thing, there are other ways of doing that thing.

for example, this scores about 25X faster than method1 in my quick estimation:

Code:

			function method3(){
				start =  new Date();
				testString=Array(numIterations+1).join('aa bb');
				end = new Date();
				$('#testResult1').val(end-start);
				return end-start;
			}

25:1 pretty much makes the differences between method1 and method2 insignificant.

What's interesting to me is watching how browsers adapt to code over time. The chrome team has jQuery in their dev-cycle nightly perf test batches. What that means is that chrome is developed to run jquery et al faster.

while twiddling knobs in an attempt to get faster real-world code, sometimes browser makers obsolete old best practices.

for example. IE and older firefox is several times faster at populating arrays using array[array.length]=value than array.push(value).

but the chome saw that more code actually uses push().

as a result, here's how chrome scores those two methods:

Code:

MS   TEST
60	array build - [].length trick
32	array build - push()

given how much faster chrome is than ie, i still use [].length, but it's an illustrative example of how everything from duff's devices, early declaration, array length caching and other common performance tweaks don't always result in millisecond payday anymore...

NadAngel · 10-16-2012, 10:24 AM

Very informative reply. I was actually trying to test the performance of generating a comma delimited string with two methods. My mistake was that I used constant data "aa bb" to generate the string that may have set you in the wrong direction. Real data would not have been constant, so you have to use a for loop like I did in my two methods. In version 2 my script uses pre-generated random data to build the string. In this case the difference between browsers is not so obvious. My quick tests show, that method2 is more efficient in all tested browsers except Firefox.

An active demo is here:
http://benjaminpajk.host22.com/test2.html

Code:

<!doctype html>
<!--Author: Benjamin Pajk-->
<!-- version 2 -->
<html lang="sl">
	<head>
		<meta charset="UTF-8">
		<title>Benchmark Test</title>
		<meta name="description" content="javaScript benchmark test">

		<meta http-equiv="X-UA-Compatible" value="IE=9"> 
		
		<!--
		<link rel="stylesheet" href="css/style.css">
		-->
		
		<script src="http://code.jquery.com/jquery-latest.js"></script>
		
		<script type="text/javascript">
		$(document).ready(function(){
				
			var numTests=50; //number of benchmark test samples (50)
			var numIterations=1048576; //one mega (10*1024*1024=1048576) iterations
		
			var testString='';
			var testData=new Array();
			var start;
			var end;
			var i;
		
			var testArray=new Array();
		
			function methodObj(){
				this.method1=-1;
				this.method2=-1;
			}
			
			function dataObj(){
				this.data1=-1;
				this.data2=-1;
			}
		
			/*perform benchmark test for bulding comma delimited string*/
			//method 1 checks if we need to insert the comma on every iteration
			function method1(){
				testString='';
				start =  new Date();
				insertComma=false;
				for(i=0; i<numIterations; i++){
					if (insertComma){
						testString+=',';
					}else{
						insertComma=true;
					}
					testString+=testData[i].data1+' '+testData[i].data2;
				}
				end = new Date();
				$('#testResult1').val(end-start);
				return end-start;
			}
			
			//method 2 inserts the comma on every iteration and removes the last comma when it finishes
			function method2(){
				testString='';
				start =  new Date();
				for(i=0; i<numIterations; i++){
					testString+=testData[i].data1+' '+testData[i].data2+',';
				}
				testString=testString.slice(0,testString.length-1);
				end = new Date();
				$('#testResult2').val(end-start);
				return end-start;
			}
			
			//test function (recursive)
			function cmp(x){
				if (x>0){
					//recursive call with 500 ms delay
					setTimeout(function(){cmp(x-1)}, 500);
				
					//create an array of random test data with numIterations elements
					testData=[];
					for(i=0; i<numIterations; i++){
						var newData=new dataObj();
						newData.data1=Math.floor(Math.random()*10);
						newData.data2=Math.floor(Math.random()*10);
						testData.push(newData);
					}
					
					var obj=new methodObj();
					obj.method1=method1();
					obj.method2=method2();
					testArray.push(obj);
					$('#parallelTest').text('Testing: '+Math.round((1-x/numTests)*100)+' %');
				}else{
					//calculate the average
					var sumArr1=0;
					var sumArr2=0;
					for(var k=0; k<testArray.length; k++){
						sumArr1+=testArray[k].method1;
						sumArr2+=testArray[k].method2;
					}
					//display average results
					$('#avgResult1').val(sumArr1/testArray.length);
					$('#avgResult2').val(sumArr2/testArray.length);
					$('#testResult1').val('');
					$('#testResult2').val('');
					$('#parallelTest').text('Start parallel test');
				}
			}
			
			$('#parallelTest').click(function(){
				//reset the variables
				testArray=[];
				$('#avgResult1').val('');
				$('#avgResult2').val('');
				//call the benchmark test function
				cmp(numTests);
			});

		});

		</script> 
		
	</head>
	<body>
		<p>
		This script tests the performace of two methods for building comma delimited strings. Each test iterates 1048576 (1024*1024) times and adds pre-generated random data 
		in format "a b" with a comma delimeter on every iteration. The test is performed 50 times for both methods and the average time per test is calculated for both methods.
		The results are expressed in milliseconds.<br>
		</p>
		<p>
		Data example: "4 5,3 8,9 1,3 5"<br>
		</p>
		<p>
		<b>method 1:</b> checks if we need to insert the comma on every iteration<br>
		<b>method 2:</b> inserts the comma on every iteration and removes the last comma when it finishes<br>
		</p>
		<p>
		Method 1:<input type="text" id="testResult1" readonly="readonly" value="">ms<br>
		Method 2:<input type="text" id="testResult2" readonly="readonly" value="">ms<br>
		Method 1 (average):<input type="text" id="avgResult1" readonly="readonly" value="">ms<br>
		Method 2 (average):<input type="text" id="avgResult2" readonly="readonly" value="">ms<br>
		<button id="parallelTest">Start parallel test</button>
		</p>
		
		<p>
		Preliminary results show, that IE9 is in favor of the first method while Chrome Firefox and Opera are in favour of the second method.<br>
		Tested browsers:<br>
		Internet Explorer 9.0.7<br>
		Firefox 16.0<br>
		Chrome (ver. 22.0.1229.94 m)<br>
		Opera 12.02<br>
		</p>
	</body>
</html>