I have an sqlite database full of huge number of URLs and it\'s taking huge amount of diskspace, and accessing it causes many disk seeks and is slow. Average URL path length is
Do you usually do a "range scan" or unique id lookup only?
If you won't do something like WHERE url like "/xxxx/question/%". You could use a hashed index rather then a b-tree index on varchar() to reduce the number of disk seeks.