Yep, in double quotations if you have a complex type you should use braces around them. A single dimension won't require them, but multiple dimensions will as the parser is ungreedy within the double quotes. So it would resolve first $_SESSION['cart'], then take the result (which is Array) and offset the [$index]. So it attempts to write it as
Array[$index] which of course will not dereference to any valid value as its not a variable. The alternative above is the approach I would use, although typically I just use printformatting so I don't need to string concat. Another alternative is to use prepared statements and bind the variables.