Reasoning brought to Granite

TLDR; What means “Reasoning” in a LLM?

“Reasoning” the ability of AI models to logical process of information, break down complex problems, and make decisions based on context and patterns.

There are lots of sites describing the reasoning in the context of LLMs and Generative AI. Reasoning will let the users to achieve way better results while they use GenAI tools in their businesses.

What is Granite

IBM Granite is a series of decoder-only AI foundation models.

IBM brought reasoning to Granite

IBM just announced the reasoning in preview mode to the Granite model family (ibm-granite/granite-3.2–8b-instruct-preview).

This feature could be tested easily as the model is available on Hugging Face site.

Testing the preview reasoning capacity with granite-3.2–8b-instruct-preview

On the HF page there is a snippet provided for the model.

from transformers import AutoModelForCausalLM, AutoTokenizer, set_seed
import torch

model_path="ibm-granite/granite-3.2-8b-instruct-preview"
device="cuda"
model = AutoModelForCausalLM.from_pretrained(
        model_path,
        device_map=device,
        torch_dtype=torch.bfloat16,
    )
tokenizer = AutoTokenizer.from_pretrained(
        model_path
)

conv = [{"role": "user", "content":"How do I develop a skill?"}]

input_ids = tokenizer.apply_chat_template(conv, return_tensors="pt", thinking=True, return_dict=True, add_generation_prompt=True).to(device)

set_seed(42)
output = model.generate(
    **input_ids,
    max_new_tokens=8192,
)

prediction = tokenizer.decode(output[0, input_ids["input_ids"].shape[1]:], skip_special_tokens=True)
print(prediction)

In order to make it runnable on my laptop, I wrapped it into a sample app built using “streamlit”.

from transformers import AutoModelForCausalLM, AutoTokenizer, set_seed
import torch
import streamlit as st

# Model and tokenizer loading (outside the Streamlit app for efficiency)
model_path = "ibm-granite/granite-3.2-8b-instruct-preview"
device = "cuda" if torch.cuda.is_available() else "cpu"  # Check for CUDA availability

try:
    model = AutoModelForCausalLM.from_pretrained(
        model_path,
        device_map=device,
        torch_dtype=torch.bfloat16,
        trust_remote_code=True # Add this if needed by the model
    )
    tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
except Exception as e:
    st.error(f"Error loading model: {e}")
    st.stop()  # Stop execution if model loading fails


st.title("🪨 Granite 3.2 Chatbot with Reasoning 🤖")

user_input = st.text_area("Enter your message:", height=150)

if st.button("Generate Response"):
    if not user_input:
        st.warning("Please enter a message.")
    else:
        conv = [{"role": "user", "content": user_input}]

        with st.spinner("Generating response..."):  # Show a spinner while generating
            try:
                input_ids = tokenizer.apply_chat_template(
                    conv, return_tensors="pt", thinking=True, return_dict=True, add_generation_prompt=True
                ).to(device)

                set_seed(42)  # For reproducibility (optional)
                output = model.generate(
                    **input_ids,
                    max_new_tokens=8192,  # Adjust as needed
                )

                prediction = tokenizer.decode(output[0, input_ids["input_ids"].shape[1]:], skip_special_tokens=True)
                st.write("Response:")
                st.write(prediction)  # Display the response

            except Exception as e:
                st.error(f"Error generating response: {e}")


# Optional: Display model information
if st.checkbox("Show Model Info"):
    st.write(f"Model: {model_path}")
    st.write(f"Device: {device}")

The required Python installation are the following (or use a requirements.txt file);

pip install torch torchvision torchaudio
pip install accelerate
pip install transformers
pip install streamlit

Disclaimer: on a CPU machine it could take forever to have an output! 😪

The output with the options of “thinking=true” would be (depending on the input)

Here is my thought process:

1. Understand the user's intent: The user is seeking guidance on how to develop a new skill. 
2. Break down the process: Skills development involves several stages - identification, planning, practice, feedback, and refinement. 
3. Consider different aspects: Discuss various methods, mindset, and resources needed for effective skill acquisition. 
4. Provide actionable steps: Offer a structured approach that the user can follow. 

Here is my response:

Developing a new skill is a systematic process that involves several key steps and a dedicated mindset. Here's a comprehensive guide to help you effectively cultivate a new skill:

### 1. **Identify the Skill**
   - **Clarity of Purpose**: Clearly define what skill you want to acquire. Be specific about why you want this skill and how it aligns with your goals or interests.
   - **Research**: Understand the basics of the skill. Read about it, watch tutorials, or talk to people who are already proficient. This initial research will give you a realistic understanding of what the skill entails.

### 2. **Set Clear Goals**
   - **SMART Goals**: Make sure your goals are Specific, Measurable, Achievable, Relevant, and Time-bound. For example, instead of saying "I want to learn programming," set a goal like "I will complete a beginner's Python course within three months."
   - **Break Down**: If the skill is complex, break it down into smaller, manageable milestones. 

### 3. **Create a Learning Plan**
   - **Resources**: Identify the best resources for learning. These could be online courses, books, mentors, workshops, or practical projects.
   - **Schedule**: Dedicate time in your routine for skill practice. Consistency is crucial in skill development. Treat your learning time as non-negotiable.

### 4. **Start with the Basics**
   - **Foundational Knowledge**: Begin by grasping the fundamentals. Understanding the basics thoroughly sets a strong foundation for more advanced learning.
   - **Practice Regularly**: Regular, focused practice is essential. Even 15-30 minutes a day can yield significant results over time.

### 5. **Seek Feedback and Guidance**
   - **Mentorship**: If possible, find a mentor or join a community of learners. Feedback from experienced individuals can accelerate your learning curve.
   - **Peer Learning**: Engage with others learning the same skill. Peer groups provide support, motivation, and different perspectives.

### 6. **Apply the Skill in Real-World Scenarios**
   - **Practical Application**: Look for opportunities to apply what you've learned. This could be through projects, internships, or volunteering. Practical application solidifies your understanding and builds confidence.
   - **Iterate Based on Feedback**: After applying your skill, seek and incorporate feedback to refine and improve.

### 7. **Maintain a Growth Mindset**
   - **Embrace Challenges**: Understand that setbacks and failures are part of the learning process. Maintain a positive attitude and view them as opportunities to learn.
   - **Patience and Persistence**: Skill development takes time. Stay patient and persistent, even when progress seems slow.

### 8. **Track and Reflect**
   - **Progress Tracking**: Regularly review your progress. This could be through keeping a learning journal or using apps designed for skill tracking.
   - **Reflect and Adjust**: Periodically reflect on your learning process. What's working? What's not? Adjust your plan accordingly.

### 9. **Stay Motivated**
   - **Reward System**: Set small rewards for achieving milestones to keep yourself motivated.
   - **Stay Curious**: Keep your interest piqued by exploring related topics or advanced aspects of the skill.

By following these steps and maintaining a disciplined, growth-oriented approach, you can effectively develop any new skill. Remember, the key is consistent effort and a willingness to learn from both successes and failures. 

Building skills is an ongoing journey, not a destination.

and with “thinking=false”

Developing a skill involves a combination of learning, practice, and often, feedback. Here's a step-by-step guide to help you develop a new skill:

1. **Identify the Skill**: Start by clearly defining what skill you want to develop. Be specific. Instead of saying "I want to learn to code," specify a programming language like Python or JavaScript.

2. **Research**: Learn about the basics of the skill. Read books, articles, watch tutorials, or take online courses. Websites like Coursera, Udemy, Khan Academy, and YouTube can be great resources.

3. **Set Clear Goals**: Break down your skill into smaller, manageable goals. For example, if you're learning a new language, your goals might be to learn basic grammar, build a simple sentence, have a basic conversation, etc.

4. **Create a Study Plan**: Allocate specific time each day or week for learning and practicing. Consistency is key in skill development.

5. **Practice**: Apply what you've learned. Practice makes permanent. If you're learning to code, write small programs. If it's a musical instrument, play regularly.

6. **Get Feedback**: Seek feedback from others who are more experienced. This could be a mentor, a tutor, or even online communities. Constructive criticism can help you identify areas for improvement.

7. **Review and Refine**: Regularly review what you've learned. Refine your skills based on feedback and your own observations.

8. **Apply in Real Life**: Try to use your new skill in real-life situations. This could be a project at work, a personal hobby, or volunteering.

9. **Be Patient and Persistent**: Skill development takes time. Don't get discouraged by slow progress or setbacks. Keep practicing and learning.

10. **Stay Motivated**: Keep your end goal in mind and celebrate small victories along the way to stay motivated.

Remember, everyone learns at their own pace, so don't compare your progress with others. The most important thing is that you're consistently moving forward.

Conclusion

I think we should stay tuned with the capacities of reasoning combined with powerful ai agents solving a lots of business cases…

Useful links

• Understanding Reasoning LLMs: https://magazine.sebastianraschka.com/p/understanding-reasoning-llms (by Sebastian Raschka, PhD)
• Bringing reasoning to Granite: https://www.ibm.com/new/announcements/bringing-reasoning-to-granite
• Hugging Face model page and information: https://huggingface.co/ibm-granite/granite-3.2-8b-instruct-preview