You could add an additional couple of lines to hide the other panel, and use the callback on the hide to call the slideToggle:
Code:
$(document).ready(function(){
$(".page-item-4").click(function(){
$("#slide-panel-video").slideUp('slow',function(){
$("#slide-panel-photo").slideToggle("slow");
});
});
$(".page-item-6").click(function(){
$("#slide-panel-photo").slideUp('slow',function(){
$("#slide-panel-video").slideToggle("slow");
});
});
});
This is OK if you've just got a couple of panels, as now, but will quickly get tedious with any more.
If you do intend to have more than one I'd redo your html so that the lists within the sidepanel are nested within the top level lists - so it becomes more like a standard drop down menu, then add some jQuery for the animation - it'll take a bit more effort initially but might be easier in the long term.