How to add external header files during bazel/tensorflow build

Question

I am trying to add external header file (like OpenCL header file) for some experimentation for tensorflow. I tried to add this into BUILD file under tensorflow/core/BUILD fi

Answer 1

I've faced with the similar problem when I built TensorFlow with Intel MKL and had to add MKL headers. My solution is the following:

1. Create symlink to your headers into third_party folder, like:

   <your tensorflow folder>/third_party/opencl/include -> /opt/OpenCL/include
   

   with command:

   ln -s /opt/OpenCL/include <your tensorflow folder>/third_party/opencl
   

2. Create simple BUILD file into <your tensorflow folder>/third_party/opencl folder:

   cc_library(
       name = "opencl",
       hdrs = glob(["include/CL/*.h"]),
       visibility = ["//visibility:public"],
   )
   

3. Add deps into target library:

   cc_library(
       name = "all_kernels",
       visibility = ["//visibility:public"],
       copts = tf_copts() + ["-Ithird_party/opencl/include"],
       deps = [
           "//third_party/opencl", 
           ...
       ],
   )
   

4. Don't forget to add compiler options either into target library as shown above or just as a flag to bazel:

    bazel build --copt="-Ithird_party/opencl/include" ...
   

Answer 2

You have to add the files as a dependency of this rule.

IIUC, you have the following structure:

tensorflow/
  BUILD
  WORKSPACE
  tensorflow/
    core/
      BUILD
  third_party/
    include -> /opt/opencl/CL # or something like that

You want to expose the .h files in a way Bazel can understand/depend on, so open up the tensorflow/BUILD file and add the following:

cc_library(
    name = "opencl",
    hdrs = glob(["third_party/include/*.h"]),
    visibility = ["//visibility:public"],
)

This creates a C++ library from the .h files under third_party/include, which can be depended on from anywhere in the source tree.

Now go to your tensorflow/core/BUILD file and add a dependency to the cc_library there:

cc_library(
    name = "all_kernels",
    visibility = ["//visibility:public"],
    copts = tf_copts() + ["-Ithird_party/include"],
    deps = [
        "//:opencl", 
        # plus any other deps
    ],
)

Setting copts just changes the flags when gcc is run. Adding //:opencl to the dependencies tells Bazel to make those files available when gcc is run.

Answer 3

Bazel tries to be very strict about making sure its builds only include files that it knows about, to try to make sure that they are reproducible. Unfortunately that can make it tough to experiment with. The correct way to solve the problem is to create a BUILD file and rule for the headers you want to include. You may be able to hack something by messing around with the bazel-* generated folders too, but I don't recommend it.