问题
I use the code to add a picture texture in RealityKit and it works fine.
var material = SimpleMaterial()
material.baseColor = try! .texture(.load(named: "image.jpg"))
I try to use this code to add a video file to be a texture, but it crashes!!!
guard let url = Bundle.main.url(forResource: "data", withExtension: "mp4") else {
return
}
material.baseColor = try! .texture(.load(contentsOf: url))
How can I add a video file?
回答1:
You can use video textures only in RealityKit 2.0
(Xcode 12 with iOS 14 target is needed). A previous version of RealityKit doesn't support video materials.
Here's a code showing you how to apply it:
import AVKit
import RealityKit
@IBOutlet var arView: ARView!
// AVPLAYER
guard let pathToVideo = Bundle.main.path(forResource: "video", ofType: "mp4")
else {
return
}
let videoURL = URL(fileURLWithPath: pathToVideo)
let avPlayer = AVPlayer(url: videoURL)
// ENTITY
let mesh = MeshResource.generatePlane(width: 1.92, depth: 1.08) // 16:9 video
let material = VideoMaterial(avPlayer: avPlayer)
let planeEntity = ModelEntity(mesh: mesh, materials: [material])
// ANCHOR
let anchor = AnchorEntity(.plane(.vertical,
classification: .wall,
minimumBounds: [0.3, 0.3])
anchor.addChild(planeEntity)
arView.scene.anchors.append(anchor)
// PLAYBACK
avPlayer.play()
Also, you can add VideoMaterial
this way:
// AVPLAYER and PlayerItem
let url = Bundle.main.url(forResource: "aaa",
withExtension: "mp4")
let asset = AVAsset(url: url!)
let playerItem = AVPlayerItem(asset: asset)
let avPlayer = AVPlayer()
// ENTITY
let mesh = MeshResource.generateSphere(radius: 1)
let material = VideoMaterial(avPlayer: avPlayer)
let entity = ModelEntity(mesh: mesh, materials: [material])
// ANCHOR
let anchor = AnchorEntity(world: [0,0,-10])
anchor.addChild(entity)
arView.scene.anchors.append(anchor)
// PLAYBACK
avPlayer.replaceCurrentItem(with: playerItem)
avPlayer.play()
回答2:
I found the workaround, such as below code
self.arView = arView
let scene = SCNScene()
scnView = SCNView(frame: arView.frame)
scnView?.scene = scene
scnView?.backgroundColor = UIColor.clear
scnView?.scene?.background.contents = UIColor.clear
Then add SCN camera and set camera transform from ARFrame, such as:
let rotation = SCNMatrix4MakeRotation(.pi / 2.0, 0, 0, 1)
let cameraTransform = simd_mul(frame.camera.transform, simd_float4x4(rotation))
let projectionMatrix = SCNMatrix4(frame.camera.projectionMatrix(for: .portrait,
viewportSize: self.viewBounds!.size,
zNear: 0.001,
zFar: 10000) )
self.arCameraNode?.simdTransform = cameraTransform
self.arCameraNode?.camera?.projectionTransform = projectionMatrix
Finally, add your SCN video node into rootNode
However, there is a little shifting, I think you can wait for Realitykit to support video material.
回答3:
I might be wrong, but currently RealityKit does not support videos. A video is not a normal texture, it is a set of animated textures.
来源:https://stackoverflow.com/questions/58131206/realitykit-how-to-add-a-video-material-to-a-modelentity