If I find (or create) a new PowerShell cmdlet (or function), how do I add it to my machine?
You should access the cmdlets through your profile script. That way, every time you access PowerShell, it gets loaded. See The Power of Profiles.
As Alex mentions, any function defined in your profile or in a script that gets "dotted" into your profile will always be available. The same goes if you use Add-PSSnapin in your profile to add a snapin. The cmdlets in the snapin will always be available. For more information about profiles check out the help topic:
man about_profiles
However if you have a significant number of functions you may not want to load them until they are needed. In this case, you can organize functionality into scripts and then put those scripts into one or more directories that are in your path. You can then reference the script by name without specifying the full path or even the .PS1 extension. For more information about using scripts check out the help topic:
man about_scripts
PowerShell V2 introduces an even better approach to organizing functions and loading them on demand. The feature is called Modules and allows you to Import-Module by a simple name (rather than path) and to choose which functions and variable are made public versus which ones remain private. If you have V2, check out modules:
man about_modules