0

When I use the default tensorflow build I get Illegal instruction core dumped error.

Based on my research https://stackoverflow.com/questions/60858317/how-to-fix-illegal-instruction-core-dumped & https://github.com/tensorflow/tensorflow/issues/17411 I need to build tensorflow from source.

I started to build tensorflow from source based on: https://www.tensorflow.org/install/source

When I would like to run ./configure I was getting an error:

Could not find any cuda.h matching version '10' in any subdirectory:
    ''
    'include'
    'include/cuda'
    'include/*-linux-gnu'
    'extras/CUPTI/include'
    'include/cuda/CUPTI'
    'local/cuda/extras/CUPTI/include'
of:
    '/lib'
    '/lib/i386-linux-gnu'
    '/lib/x86_64-linux-gnu'
    '/usr'
    '/usr/lib/x86_64-linux-gnu/libfakeroot'

Therefore I installed cuda toolkit and cudnn based on this post: https://towardsdatascience.com/installing-tensorflow-gpu-in-ubuntu-20-04-4ee3ca4cb75d

now I have:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243

and Cudddn:

cudnn-10.1-linux-x64-v7.6.5.32 

my cuda is in:

whereis cuda
cuda: /usr/lib/cuda /usr/include/cuda.h

and nvidia-smi returns:

nvidia-smi

Now I would expect that I'd be able to run ./configure I get the following messages:

WARNING: current bazel installation is not a release version.
Make sure you are running at least bazel 3.7.2
Please specify the location of python. [Default is /usr/bin/python3]: 


 Found possible Python library paths:
   /usr/lib/python3/dist-packages
    /usr/local/lib/python3.8/dist-packages
   Please input the desired Python library path to use.  Default is [/usr/lib/python3/dist-packages]

  Do you wish to build TensorFlow with ROCm support? [y/N]: 
  No ROCm support will be enabled for TensorFlow.

  Do you wish to build TensorFlow with CUDA support? [y/N]: y
  CUDA support will be enabled for TensorFlow.

  Do you wish to build TensorFlow with TensorRT support? [y/N]: 
  No TensorRT support will be enabled for TensorFlow.

  Inconsistent CUDA toolkit path: /usr vs /usr/lib
  Asking for detailed CUDA configuration...


 Please specify the CUDA SDK version you want to use. [Leave empty to default to CUDA 10]: 


 Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 7]: 


 Please specify the locally installed NCCL version you want to use. [Leave empty to use 
 http://github.com/nvidia/nccl]: 


 Please specify the comma-separated list of base paths to look for CUDA libraries and headers. 
 [Leave empty to use the default]: 

but can not pass this error:

Inconsistent CUDA toolkit path: /usr vs /usr/lib Asking for detailed CUDA configuration...

What is wrong? how do I resolve this problem?

1 Answer 1

0

I had the same issue when trying to build Deepspeech, and solved it by reinstalling CUDA from the nvidia repository.
As an example, if you want to install CUDA 11.3 on Ubuntu 20.04 and over the network, you have to enter the following commands:
$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin
$ sudo mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600
$ sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/7fa2af80.pub
$ sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /"
$ sudo apt-get update
$ sudo apt-get -y install cuda

Source: https://developer.nvidia.com/cuda-11.3.0-download-archive.

In my case these instructions installed CUDA under /usr/local/ and not under /usr/ directly. This way the configuration process was able to find the CUDA installation.

I hope this helps someone.

1
  • While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - From Review
    – David
    Jan 11 at 5:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.