How to Implement a Voice Command System in FlutterFlow

In FlutterFlow, go to Settings → Pubspec Dependencies and add speech_to_text: ^6.6.2 and flutter_tts: ^3.8.5. Then configure the required platform permissions. In FlutterFlow's Settings → App Permissions panel, enable Microphone. For iOS, this adds the NSMicrophoneUsageDescription and NSSpeechRecognitionUsageDescription keys to Info.plist automatically. For Android, the RECORD_AUDIO and INTERNET permissions are added to AndroidManifest.xml. The speech_to_text package requires network access on Android because it sends audio to Google's on-device speech recognition service.

Create two Custom Actions. The first, startListening, initialises a SpeechToText instance, calls speechToText.initialize(), and then calls speechToText.listen() with an onResult callback. In the callback, when finalResult is true, store the recognised text in a non-persisted App State variable named lastVoiceCommand. The second action, stopListening, calls speechToText.stop() and updates a Boolean App State variable isListening to false. Store the SpeechToText instance as a static variable so both actions share the same instance. Call startListening from the mic button's On Press action and stopListening from the same button when isListening is true.

1// Custom Action pair: startListening + stopListening
2import 'package:speech_to_text/speech_to_text.dart';
3
4// Shared instance
5final _speech = SpeechToText();
6bool _isInitialized = false;
7
8// Action 1: startListening
9// No parameters. Updates App State: lastVoiceCommand (String), isListening (Bool)
10Future<void> startListening(
11  Future<void> Function(String) onCommandRecognized,
12) async {
13  if (!_isInitialized) {
14    _isInitialized = await _speech.initialize(
15      onError: (err) => print('Speech error: ${err.errorMsg}'),
16    );
17  }
18  if (!_isInitialized) return;
19
20  await _speech.listen(
21    onResult: (result) {
22      if (result.finalResult) {
23        onCommandRecognized(result.recognizedWords.toLowerCase());
24      }
25    },
26    listenFor: const Duration(seconds: 10),
27    pauseFor: const Duration(seconds: 3),
28    localeId: 'en_US',
29  );
30}
31
32// Action 2: stopListening (no parameters)
33Future<void> stopListening() async {
34  await _speech.stop();
35}

Create a Custom Action named processVoiceCommand that accepts the recognized text string and returns a command identifier string. Inside the action, define a command dictionary as a Map where keys are command identifiers and values are lists of trigger keywords. For each entry in the dictionary, check if the input text contains any of the trigger keywords using String.contains(). Return the first matching command identifier, or 'unknown' if no keywords match. In your FlutterFlow Action Flow, after startListening sets the App State variable, call processVoiceCommand with the transcript, then use a Switch or series of Conditional Actions to navigate or trigger the appropriate action based on the returned command identifier.

1// Custom Action: processVoiceCommand
2// Parameters: transcript (String)
3// Returns: String (command identifier)
4Future<String> processVoiceCommand(String transcript) async {
5  final text = transcript.toLowerCase().trim();
6
7  // Command dictionary: id -> trigger keywords (any match triggers the command)
8  final commands = <String, List<String>>{
9    'navigate_home':     ['home', 'main', 'dashboard', 'start'],
10    'navigate_profile':  ['profile', 'account', 'settings', 'my account'],
11    'navigate_orders':   ['order', 'orders', 'purchase', 'bought'],
12    'navigate_search':   ['search', 'find', 'look for', 'browse'],
13    'action_back':       ['back', 'go back', 'previous', 'return'],
14    'action_refresh':    ['refresh', 'reload', 'update'],
15    'action_help':       ['help', 'assist', 'what can', 'commands'],
16  };
17
18  for (final entry in commands.entries) {
19    for (final keyword in entry.value) {
20      if (text.contains(keyword)) {
21        return entry.key;
22      }
23    }
24  }
25
26  return 'unknown';
27}

Create a Custom Action named speakFeedback that accepts a message string and uses the flutter_tts FlutterTts class to speak it aloud. Call flutterTts.setLanguage('en-US'), flutterTts.setSpeechRate(0.5) for a natural speaking pace, and flutterTts.speak(message). In your command processing flow, call speakFeedback after identifying the command: 'Going to Home' for navigate_home, 'Showing your orders' for navigate_orders, and 'Sorry, I did not understand that. Try saying home, orders, or help.' for unknown. Spoken feedback is essential because users' eyes are often elsewhere when using voice control — they need audio confirmation that their command was heard.

Create a FloatingActionButton in your app's main scaffold. Bind its icon to the isListening App State variable: show Icons.mic_none when not listening and Icons.mic when listening. Add a Container behind the button that scales from 1.0 to 1.3 and pulses red when isListening is true, using FlutterFlow's built-in animation settings on the Container. Connect the FAB's On Tap action to a Conditional Action: if isListening is false, call startListening and set isListening to true; if isListening is true, call stopListening and set isListening to false. This gives users clear visual feedback that the app is actively capturing their voice.