You forgot to quantify the regex, it also wasn't quite properly formed.
public function store(Request $request){
$this->validate($request, ['filename' => 'regex:/^[a-zA-Z0-9_\-]*$/']);
}
This will accept empty filenames; if you want to accept non-empty only change the *
to +
.