Recently did rockingham, it's oop norf so not too far away for you.
I like you was dropping bricks thinking about binning it.
I was only able to do it as I have recently moved so no longer need the bike to commute, so if I binned I would be able to take my time fixing the bike up.
Depends on your own circumstances as to what you can manage.
In terms of worrying, if you have seen the other thread recently asking if a crash was his fault, take a look at the comments. You can mitigate the risks somewhat by riding for yourself and staying in your limits, you can guarantee that you won't crash due to others actions but as I say, it's about mitigating the risks.
I would say anything with a lot of long straights wouldn't be great on the sv, unless there is also some good twisty sections, that's where you'll really enjoy it.
In terms of doing a track specific bike, I'd say not worth it at the moment. Get some experience under you on the sv both on road and on track, learn the bike, it's capabilities and limits and your own. After that look at doing a track bike.
Well that's my plan at least, got 4 and a bit years road experience, did one track day and learnt more about the sv's abilities in 3 sessions than I did in the 4 years.
Other than all that stuff, just enjoy yourself and take your time.
If you can get a mate to go with you, you will have some company. Also, if you can take something to sit on!